Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
BusigencePosted 14 days ago

PySpark Developer

Who we'd reach at BusigenceFound and reached by Power Apply
Best leadEngineering manager

Leads the team you would join

Technical recruiter

Owns the opening

Team member

On the team, can refer you

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a full-time, remote, immediate-start role open worldwide, for early-career engineers who build data and machine learning pipelines in PySpark. You would fetch data from databases, APIs and files, and build distributed Python logic for data science applications at scale. The posting asks for production PySpark experience, performance tuning and the ability to learn PySpark MLlib quickly. The company is an AI and data products firm that describes an accelerated hiring process.

What you’ll do

  • Fetch data from databases, APIs and flat files.
  • Convert and distribute Python code into functional programming style for PySpark.
  • Build complex mathematical logic in PySpark across parallel clusters.
  • Apply performance tuning, optimisation, configuration and scheduling in PySpark.
  • Integrate APIs, streams, databases and files such as JSON, XML and CSV through PySpark.
  • Work with the data science team on data applications.

What they’re looking for

  • Strong Python 3 skills, including NumPy and Pandas.
  • PySpark Core and PySpark SQL skills.
  • Advanced SQL.
  • Experience building data engineering pipelines on real-world problems.
  • Experience with at least one production PySpark project.
  • Experience with performance tuning and avoiding runtime bottlenecks.
  • Ability to learn PySpark MLlib within two weeks.
  • Strong understanding of data structures, linear algebra and algorithms.

Nice to have

  • Working knowledge of functional programming concepts such as higher-order functions, lazy evaluation and immutable data.
  • A firm grasp of the mathematics behind modelling techniques for large datasets of over one million records.
  • Experience with PySpark MLlib.
  • Experience configuring checkpointing and DAGs on a Spark cluster.
  • Experience developing a data platform.

What’s on offer

  • Remote, work-from-anywhere arrangement.
  • Compensation stated as above industry standards.
  • Flexible working hours and a startup culture.
  • Learning opportunities.

Skills

Questions about this role

Is this PySpark Developer role remote?

Yes. The role is remote and works from anywhere, and the posting asks for immediate joiners.

How many years of experience are needed?

The posting asks for zero to four years of relevant experience.

Does the posting mention salary?

The posting says compensation is above industry standards, without giving a figure.