Leads the team this role joins
Hires for this team
On the team, can refer you
From your own Gmail and LinkedIn.
About the role
This is a senior, full-time, on-site role in Hyderabad, India, for a data engineer who will move and stabilise batch and streaming pipelines on Databricks and AWS. The role sits within S&P Dow Jones Indices, a division of S&P Global, and supports a global data mastering effort with offshore delivery teams. The engineer will also provide technical guidance to peers and use AI-assisted tools in day-to-day engineering.
What you’ll do
- Help move existing pipelines onto the target Databricks data platform.
- Stabilise and optimise pipelines after migration for performance and reliability.
- Apply repeatable patterns for ingestion, transformation, testing, deployment and monitoring.
- Support secure AWS deployments with IAM, encryption and governed data access.
- Onboard new datasets using standard ingestion and transformation frameworks.
- Help integrate platform datasets with the enterprise data mastering platform, including quality and reconciliation controls.
- Support golden record maintenance for key financial datasets where the programme requires it.
- Review code and designs, and help improve CI/CD, infrastructure automation and runbooks.
- Act as a technical link between offshore teams and global platform stakeholders.
What they’re looking for
- 5 to 7 or more years of experience in data engineering or data platforms.
- Hands-on experience building and running batch or streaming pipelines.
- Experience with Databricks and lakehouse architectures.
- Working knowledge of AWS data services such as S3, Glue, Lambda, Lake Formation, Kinesis and IAM.
- Working knowledge of Delta Lake, Apache Iceberg and Unity Catalog or equivalent.
- Understanding of data quality, schema management, lineage, access control and encryption.
- Ability to guide engineers without managing people.
- Experience working with global teams across time zones.
- Practical use of AI engineering tools such as Claude or GitHub Copilot, with the judgement to validate their output.
Nice to have
- Experience with enterprise onboarding frameworks and reusable ingestion patterns.
- Familiarity with data cataloguing and governed data products.
- Knowledge of semantic modelling and canonical models.
- Exposure to master data management and golden record workflows.
- Experience in regulated environments with strong audit expectations.
- Databricks or AWS certifications, or equivalent practical experience.
What’s on offer
- On-site work in Hyderabad, India.
- Health care coverage and wellness support.
- Generous time off.
- Continuous learning resources.
- Retirement planning and a company-matched student loan contribution through a continuing education programme.
- Family-friendly perks and employee discounts and referral awards.
- Benefits vary by country, and details are published on the employer's benefits site.
Skills
Questions about this role
Is this role remote?
No. The role is on-site in Hyderabad, Telangana, India.
How many years of experience are needed?
The posting asks for 5 to 7 or more years of data engineering or data platform experience.
Is AI tool experience required?
Practical experience with AI-assisted engineering tools such as Claude or GitHub Copilot is listed as required.
Is a salary listed?
No. The posting does not state a salary figure.