Leads the team you would join
Owns the opening
On the team, can refer you
From your own Gmail and LinkedIn.
How this role compares
One of 78 open Senior Data Engineer roles we are tracking today.
This posting does not state pay. The middle of the 16 that do advertises £90,000 – £110,000 a year.
Posted 10 days ago, with 40 of the others posted since and 37 still open from before.
Counted across the roles Frontrunner has open right now.
About the role
This is a full-time senior data engineer role based in Noida, India, on a hybrid arrangement, for someone with at least five years of data engineering experience. The engineer builds and maintains pipelines and integrations in a Databricks-based data platform, onboarding new data sources and delivering analytics-ready datasets. The role reports to the Senior Manager for Business Insights and Analytics and works with analytics, engineering, IT and cloud, security and business teams.
What you’ll do
- Design and optimize pipelines using Databricks, Apache Spark, PySpark, Python, SQL and Delta Lake.
- Build a medallion architecture with bronze, silver and gold layers.
- Build and maintain integrations with enterprise applications, REST APIs and databases.
- Create reusable ingestion frameworks for batch and near-real-time loads, including change data capture and schema evolution.
- Configure Databricks Workflows and build pipelines with Auto Loader and Lakeflow Spark Declarative Pipelines.
- Tune Spark and Delta Lake workloads for performance and compute cost.
- Implement Unity Catalog governance, covering access control, lineage, auditing and sensitive-data masking.
- Maintain existing workflows on Amazon Redshift and Amazon EC2, working with cloud teams on IAM, S3 access and deployments.
- Apply code review, automated testing and CI/CD practices, and guide other team members.
What they’re looking for
- Five or more years in data engineering or a related role, including production pipelines.
- Strong hands-on Databricks experience.
- Strong Python, PySpark, SQL and Apache Spark skills.
- Practical Delta Lake experience, including table management and incremental processing.
- Experience with medallion architecture and building analytics-ready datasets.
- Experience with Databricks Workflows or equivalent orchestration tools.
- Experience building ingestion pipelines from REST APIs, databases and enterprise applications.
- Understanding of retries, idempotency, schema evolution, backfills and failure recovery.
- Working knowledge of AWS, particularly IAM, S3 and secure connectivity.
- Experience with Git, code reviews, automated testing and CI/CD.
- Strong problem solving and communication skills.
Nice to have
- Experience with Lakeflow Spark Declarative Pipelines, Auto Loader and advanced incremental ingestion.
- Experience implementing Unity Catalog governance, lineage and sensitive-data protection.
- Experience with Databricks Genie tools.
- Exposure to AI, machine learning, LLMs or Amazon Bedrock.
What’s on offer
- Hybrid working arrangement in Noida.
Skills
Questions about this role
Where is this job based?
In Noida, India, on a hybrid arrangement.
How many years of experience do I need?
Five or more years in data engineering or a related role.
Do I need AWS experience?
Working knowledge of AWS is required, particularly IAM, S3 access and secure connectivity.