Leads the team this role joins
In-house, owns the opening
Director or head of the group
From your own Gmail and LinkedIn.
About the role
This is a full-time senior big data engineering role on the Tech Enablement team, open to US candidates, and the company describes most of its US roles as eligible for remote work. It suits engineers with at least four years of experience building data pipelines that feed a customer-facing product and who can work with Apache Spark at scale. The team builds the shared data, ML and AI infrastructure that product teams depend on, from ingestion and orchestration to documentation and an LLM gateway.
What you’ll do
- Design, build, deploy and maintain pipelines that bring web events into the data lake and event streams.
- Handle large, changing datasets with batch and streaming methods, balancing speed, cost and reliability.
- Build frameworks that speed up ML development and improve the reliability of ML-powered products.
- Build tooling that lets people and automated processes add, maintain and find data documentation.
- Work on an agentic data exploration tool that helps teams query and analyze data from natural language.
What they’re looking for
- At least 4 years of professional software development experience, including data pipelines that feed a customer-facing product.
- Hands-on Apache Spark experience writing, tuning and debugging batch or streaming jobs at scale.
- Experience deploying and operating containerized services and batch jobs.
- Solid computer science fundamentals, including distributed systems and testing.
- A bachelor's, master's or PhD in a related field, or equivalent practical experience.
Nice to have
- AWS knowledge (S3, IAM, EC2, MSK or equivalents) and instrumenting jobs with metrics, logs and alerts.
- Data lake experience with Parquet, AWS Glue or a similar catalog, and Apache Iceberg or Delta Lake.
- Streaming experience with Kafka, Flink or Spark Structured Streaming.
- Workflow orchestration with Apache Airflow, including backfills and on-call for scheduled pipelines.
- Kubernetes and Docker deployment experience.
- Experience building a data platform for a user-facing product.
What’s on offer
- US base salary range of $190,000 to $250,000 per year, with individual pay set by location, skills and experience.
- Competitive compensation and a benefits package.
- Flexible vacation and paid time off.
- Employer-matched 401(k) plan.
- A hybrid work environment, with most US positions also open to remote work.
Skills
Questions about this role
Is this role remote?
The company offers a hybrid environment, and most US-based positions can also be done remotely.
How much does it pay?
The US base salary range is $190,000 to $250,000 per year.
How many years of experience are needed?
At least four years of professional software development experience.
Is Apache Spark required?
Yes. Hands-on Spark experience for batch or streaming jobs at scale is a minimum qualification.