Leads the team this role joins
Decides on early hires
Runs hiring at the company
From your own Gmail and LinkedIn.
How this role compares
One of 78 open Senior Data Engineer roles we are tracking today.
This posting does not state pay. The middle of the 16 that do advertises £90,000 – £110,000 a year.
Posted 17 days ago, with 61 of the others posted since and 16 still open from before.
Counted across the roles Frontrunner has open right now.
About the role
This is a remote senior data engineering role in Chennai, Tamil Nadu, India, for an engineer with five to eight years of production pipeline experience. The work covers ingestion, deduplication, reconciliation and recovery for source data that feeds a search and matching product. The role also involves working on embedding and vector search pipelines and keeping them consistent. The role is listed as full-time at mid-senior level.
What you’ll do
- Automate ingestion, validation, deduplication and incremental updates from approved sources.
- Make onboarding of new sources repeatable and reduce the backlog of validated sources.
- Build safe retries, checkpoints, replay and backfill, with quarantine of bad records and no silent loss or duplication.
- Reconcile record counts and maintain metadata, source identifiers and lineage.
- Detect schema changes, apply quality checks and investigate pipeline failures.
- Maintain matching service interfaces and contribute to agreed machine learning changes.
- Test embedding and indexing consistency and matching quality.
- Work with DevOps on logs and alerts for freshness, failures, throughput, latency and cost.
- Maintain tests, deployment practices and runbooks for independent operation and recovery.
- Apply agreed consent, access, retention and deletion rules and report any unresolved mappings.
- Document existing flows, source interfaces, transformations and failure modes during handover.
What they’re looking for
- Five to eight years owning production data pipelines, including failure investigation and recovery.
- Strong SQL and PostgreSQL data modelling skills, with practical Go orchestration and migration experience.
- Experience with APIs, batch and asynchronous processing, schema evolution, metadata and lineage.
- Working knowledge of idempotency, reconciliation and safe reruns.
- Production experience with vector search and embedding pipelines, including diagnosing input, embedding, indexing and quality problems.
- Cloud operations, automated testing, code review and deployment discipline.
- Experience with monitoring, alerts, runbooks, privacy-aware data modelling and access controls.
- Clear communication with product, engineering and DevOps teams.
Nice to have
- Direct pgvector and Amazon Bedrock experience, which is strongly preferred.
- VoyageAI embedding integrations and embedding evaluation.
- Python data processing experience.
- Entity resolution and search relevance experience.
- Recruitment technology or HR technology experience.
Skills
Questions about this role
Is this senior data engineer role remote?
Yes, the role is listed as remote, based in Chennai, India.
How many years of experience do I need?
The posting asks for five to eight years of experience owning production data pipelines.
Do I need vector search experience?
Production experience with vector search and embedding pipelines is listed as required.
Is pgvector required?
No. Direct pgvector and Amazon Bedrock experience is listed as strongly preferred, not required.