Runs the team day to day
In-house, owns the opening
Joined the team recently
From your own Gmail and LinkedIn.
About the role
This is a full-time remote research role open to candidates worldwide, building data and evaluation systems for legal AI models. You will build synthetic data pipelines, design evaluations and work with legal experts to define quality metrics. The team is small, and each person owns a large part of the data and measurement work. The role suits engineers who can build reliable data systems and question what a metric actually shows.
What you’ll do
- Build synthetic data pipelines that generate, filter and curate training and evaluation samples for legal reasoning.
- Design evaluations and verifiers that reflect real legal work and check them against expert judgment.
- Build and maintain capability benchmarks and product-focused evaluations.
- Work with legal domain experts to define quality metrics, rubrics and grading methods.
- Own training and evaluation data quality, including provenance, coverage and protection against contamination.
- Turn product usage and customer feedback into new evaluations and training data.
- Investigate model failures with researchers and separate real gains from misleading metrics.
What they’re looking for
- Experience building data pipelines, ML evaluation systems or experimentation tooling, with strong Python skills.
- A rigorous approach to measurement, including statistical reasoning and experimental design.
- Ability to turn ambiguous goals or expert judgment into structured datasets, rubrics and measurable criteria.
Nice to have
- Experience with synthetic data, LLM evaluations, automated raters or preference data.
- Data and measurement engineering experience close to ML model development, even without prior LLM work.
What’s on offer
- Full-time remote work, open to candidates worldwide.
- Compensation of 230,000 to 450,000 USD, depending on location and level.
- The employer is hiring across multiple levels for this role.
Skills
Questions about this role
Is this role remote?
Yes, the position is remote and open to candidates worldwide.
How much does this job pay?
Compensation is listed as 230,000 to 450,000 USD, depending on location and level.
Is this a full-time role?
Yes, it is listed as a full-time position.
What skills are needed?
The role asks for strong Python, experience with data pipelines or ML evaluation, and a rigorous approach to measurement.