Who we'd reach at Client of Sampoorna Consultants Pvt. LtdFound and reached by Power Apply
Best leadHiring manager
Leads the team this role joins
Team director
Heads the group this role sits in
Recruiter
In-house, owns the opening
From your own Gmail and LinkedIn.
About the role
This is a data engineering role in Hyderabad, Bengaluru or Gurugram for engineers with six to ten years of experience who have run Apache Airflow themselves and built AWS pipelines. A consultancy is placing the candidate with one of its clients, so the work is pipeline delivery on the client side with a strong focus on data validation and reconciliation. Managed-only Airflow experience does not meet the bar, and candidates with only pipeline monitoring or Spark-focused work are rated as weaker fits.
What you’ll do
- Build and maintain data pipelines using Amazon S3, AWS Glue and Amazon Athena.
- Deploy and operate self-managed Apache Airflow, covering scheduling, retries and monitoring.
- Write advanced SQL and Python for ETL, ELT and automated data checks.
- Develop and test dbt models and manage AWS infrastructure with Terraform.
- Reconcile source and target data, and investigate missing, duplicate or incorrect records.
What they’re looking for
- Hands-on Airflow installation, configuration and operations, not only work inside a managed Airflow service
- Production use of S3, Glue and Athena together
- Advanced SQL including window functions and query optimisation
- Strong Python for ETL, transformation and validation
- Data quality and reconciliation experience
- dbt and Terraform experience
- Six to ten years of relevant experience
Nice to have
- Healthcare, health-tech or personal and sensitive data background
- Knowledge of data privacy, security and compliance rules
- Experience with regulated or confidential datasets
What’s on offer
- Full-time, permanent role
- Any graduate may apply
Skills
Questions about this role
How many years of experience does this role need?
Six to ten years, according to the posting.
Which cities is this role in?
Bengaluru, Hyderabad or Gurugram.
Do I need a degree?
Any graduate may apply.