Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
Circuit CompilersPosted 17 days ago

Data Engineer - Databricks

Who we'd reach at Circuit CompilersFound and reached by Power Apply
Best leadEngineering manager

Leads the team you would join

Technical recruiter

Owns the opening

Team member

On the team, can refer you

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a full-time Databricks data engineering role in Hyderabad for someone with about 5 years of experience building pipelines on the Databricks Lakehouse. The role centres on Python, PySpark and SQL work, covering ingestion, transformation, migration from legacy platforms and orchestration of batch and streaming jobs. The role also expects you to keep the platform secure, cost-efficient and deployable through code.

What you’ll do

  • Parse, cleanse and validate data, then build and run pipelines and ETL or ELT scripts.
  • Run and support the Databricks Lakehouse, including Delta Lake, Unity Catalog, Auto Loader, Lakeflow and Databricks SQL.
  • Migrate legacy warehouses, Hadoop or Spark workloads to Databricks.
  • Consume REST APIs from notebooks and jobs using OAuth, API keys or tokens.
  • Build, schedule and debug batch and streaming jobs and declarative pipelines.
  • Design Medallion (Bronze, Silver, Gold) architectures and tune Spark jobs and Delta tables for performance and cost.
  • Apply Unity Catalog permissions, service principals and secret scopes.
  • Use CI/CD and Databricks Asset Bundles to package and promote work.

What they’re looking for

  • About 5 years of data engineering experience.
  • Expert knowledge of the Databricks Lakehouse.
  • Strong programming in Python, PySpark and SQL.
  • Hands-on migration experience from legacy warehouses, Hadoop or Spark.
  • Understanding of Databricks security and DevOps practices.

Nice to have

  • Databricks Data Engineer Associate or Professional certification.
  • Knowledge of AWS services such as S3, IAM, Lambda, API Gateway and CloudWatch.
  • Familiarity with Kafka or Kinesis streaming, Terraform, Jenkins, GitHub Actions and SonarQube.
  • Awareness of MLflow, Feature Store and Mosaic AI Model Serving.

Skills

Questions about this role

Where is this job based?

The role is based in Hyderabad.

How much experience is required?

The posting asks for 5 years of experience.

Is a Databricks certification required?

No, it is described as highly preferred but not required.

Which programming languages are needed?

Strong Python, PySpark and SQL are required.