Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
Sigma SoftwarePosted 4 days ago

Data Engineer (DATABRICKS / PYSPARK)

Who we'd reach at Sigma SoftwareFound and reached by Power Apply
Best leadEngineering manager

Leads the team you would join

Technical recruiter

Owns the opening

Team member

On the team, can refer you

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a mid-level data engineering role open to remote candidates, building batch and near real-time pipelines on Databricks for an AdTech and analytics company. It suits engineers with at least two years in big data development who know SQL, Python, PySpark and Delta Lake. The role sits with Sigma Software, a global IT consultancy, and the client runs high-load marketing data platforms.

What you’ll do

  • Develop and maintain pipelines for large volumes of data in batch and near real-time settings.
  • Transform, validate and optimise data with Databricks, PySpark and SQL.
  • Support Data Lakehouse implementations that follow engineering standards.
  • Build, test and automate ETL processes and data workflows.
  • Improve data quality, organisation and reliability across platforms.
  • Troubleshoot, optimise and monitor data pipelines.
  • Work with engineers, analysts and stakeholders on scalable solutions.

What they’re looking for

  • At least two years in big data development and database design.
  • Strong hands-on SQL.
  • Strong Python development.
  • Hands-on PySpark experience.
  • Experience building and maintaining Databricks pipelines.
  • Delta Lake experience.
  • Medallion architecture experience with Bronze, Silver and Gold layers.
  • Strong analytical and problem-solving skills.
  • Intermediate or higher English.

Nice to have

  • AWS services experience, including S3.

What’s on offer

  • Remote working.
  • Full-time employment.

Skills

Questions about this role

Is this role remote?

Yes, the posting lists a remote work mode.

How much experience is needed?

At least two years in big data development and database design.

Is AWS required?

No, AWS experience including S3 is listed as a plus.