Data Engineering Lead
Skills
About the role
This is a lead-level data engineering role at Calfus, based in Bengaluru, India. It suits a senior engineer ready to architect and own an enterprise data platform end-to-end, on Azure Databricks and/or Snowflake. The role is the senior technical voice on the team, with BI and reporting treated as downstream outputs rather than the platform itself.
What you’ll do
- Design and own end-to-end data pipeline architecture across cloud-native platforms.
- Architect Medallion (Bronze/Silver/Gold) layered data designs using Delta Lake.
- Own Unity Catalog governance, access control and data lineage.
- Oversee ETL/ELT pipeline development and orchestration, including legacy migration support.
- Own dimensional modeling and SCD implementation for historical and current-state reporting.
- Write complex SQL queries and stored procedures for large-scale data transformations.
- Ensure the platform reliably feeds downstream BI tools with clean, curated data.
- Work with stakeholders to translate requirements into technical specifications.
- Analyze and optimize Spark and pipeline workloads for performance and reliability.
- Implement RBAC, data classification and lineage tracking for compliance.
- Mentor and guide data engineers.
What they’re looking for
- 8+ years of data engineering experience with demonstrated lead-level ownership.
- Bachelor's degree in Computer Science, Information Systems, Data Science or related field.
- Deep hands-on expertise in Azure Databricks (PySpark, Delta Lake) and/or Snowflake, plus ADF or equivalent orchestration.
- Strong data modeling experience: dimensional modeling, SCD, Medallion or equivalent layered architecture.
- Strong proficiency in SQL, including advanced query writing.
- Strong Python foundation, including Pandas, NumPy and PySpark.
- Experience with data serialization formats such as JSON, CSV and Parquet.
- Pipeline orchestration experience using Airflow.
- Experience with cloud services such as S3 and AWS Lambda.
- Familiarity with cloud-native databases such as Snowflake, Postgres or Redshift.
- Full lifecycle experience from design through production support.
Nice to have
- Relevant cloud/platform certifications (Databricks, Snowflake, Azure).
- BI/reporting tool exposure such as Power BI or Tableau.
- Experience with visualization tools such as QuickSight, Plotly or Dash.
- Ability to interact with REST APIs and perform web scraping.
- Azure SDK familiarity.
What’s on offer
- Medical, group and parental insurance.
- Gratuity and provident fund options.
- Birthday leave.
Questions about this role
How many years of experience do I need?
8+ years of data engineering experience with demonstrated lead-level ownership.
Do I need a degree?
A bachelor's degree in Computer Science, Information Systems, Data Science or a related field is expected.
What platforms does this role use?
Azure Databricks and/or Snowflake, with Delta Lake, Unity Catalog and Airflow/ADF orchestration.
Related roles
Lead Data Engineer – Databricks & AI | Remote Canada
Lead data engineer role for a financial services client, requiring 10+ years of Databricks expertise and experience collaborating with AI/ML teams to build scalable data infrastructure.
Data Architect / Data Engineering Lead
A principal data architecture and engineering leadership role at UnitedHealth Group's Optum/LHI, based in Minnetonka, Minnesota. Suited to a senior data leader with deep Snowflake/Databricks and Airflow experience who wants to combine architecture ownership with hands-on delivery.
Data Analyst - Data Quality & Databricks
A data analyst role centered on data quality, profiling and reconciliation using Databricks, SQL and PySpark, open across Bangalore, Chennai and Pune for candidates with 7-11 years of experience.
Data Engineer, Data Platform & ML (Remote)
A Poland-remote Data Engineer role building a cloud-based corporate data warehouse for business analytics and machine learning. It suits a data engineer with at least three years of experience in Python, SQL, warehouse design, and modern data tooling.
Data Engineer, Data Platform & Machine Learning (Remote)
A Serbia-remote data engineering role building Tabby's corporate data warehouse, data integrations, and pipelines. It is intended for a data engineer with warehouse design, Python, SQL, and cloud data-stack experience.
Senior Data Engineer, Data Platform & ML (Remote)
A remote senior data engineering role in Yerevan focused on feature-store development, data services, streaming, and production data pipelines. It is suited to an engineer with scalable systems experience across data, machine learning, or backend development.