Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
Astra-North Infoteck Inc.Posted 3 days ago

Principal Databricks Data Engineer

Who we'd reach at Astra-North Infoteck Inc.Found and reached by Power Apply
Best leadEngineering lead

Leads the team you would join

CTO

Signs off on engineering hires

Technical recruiter

Owns the opening

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a principal-level data engineering role in Toronto, Canada, for an engineer with 12-18 years of data engineering experience who can lead a move from Cloudera to Databricks. The work centres on finance and risk data platforms, with the engineer making architecture decisions and mentoring senior engineers. The platform supports regulatory, management and analytical reporting.

What you’ll do

  • Lead Cloudera-to-Databricks Lakehouse modernization for finance and risk data.
  • Move HDFS-based ingestion, transformation and consumption onto cloud object storage and Delta Lake.
  • Convert legacy Hive and Impala logic into PySpark and Spark SQL ELT pipelines.
  • Keep data reconciliation, audit integrity and consistency intact through each migration.
  • Design and govern enterprise warehouse and lakehouse architecture, including curated and semantic layers for BI.
  • Define business metrics, hierarchies and KPIs for reporting consumers.
  • Set standards for data quality, lineage, metadata and security.
  • Act as technical authority and mentor senior engineers.

What they’re looking for

  • 12-18 years of overall data engineering experience.
  • 8+ years with enterprise data warehouse and data lake platforms.
  • 5+ years of hands-on Databricks and Apache Spark work at scale.
  • A record of modernizing legacy Cloudera platforms.
  • Experience with finance and risk data models, including financial hierarchies and liquidity risk.
  • Experience with AWS or Azure.
  • Experience with CI/CD pipelines.

Nice to have

  • Experience in BFSI, capital markets or regulatory reporting.
  • Oracle Financials experience.
  • Experience supporting AI and ML workloads.
  • Databricks or cloud certifications.
  • dbt experience, which the posting calls an advantage.

Skills

Questions about this role

How many years of experience does this role need?

The posting asks for 12-18 years of overall data engineering experience, including 5+ years of hands-on Databricks and Spark work.

Where is this job based?

The role is listed in Toronto, Canada.

Is Databricks experience required?

Yes. The posting asks for at least five years of hands-on Databricks and Apache Spark experience at scale.