Leads the team you would join
Signs off on engineering hires
Owns the opening
From your own Gmail and LinkedIn.
About the role
This is a full-time remote principal-level research and engineering role open to candidates in 55 countries, focused on evaluating AI agents and LLM systems. The role covers agent evaluations, failure analysis and rapid experimentation for enterprise workflows, working with researchers and enterprise customers. You will also build Python evaluation pipelines and research tooling that speed up experiments. Applications are completed through a link sent by email after the initial application, and are not considered until that step is done.
What you’ll do
- Design and implement evaluation frameworks, benchmarks and datasets for LLM-powered agents and complex AI workflows.
- Identify, analyse and prioritise model, agent and workflow failures in production environments.
- Run rapid experiments to test hypotheses and improve agent reliability.
- Partner with enterprise customers and researchers to evaluate, deploy and optimise AI systems.
- Develop training and evaluation data, task specifications and feedback mechanisms.
- Build Python evaluation pipelines, experimentation tools and lightweight research infrastructure.
- Define evaluation strategies and optimisation roadmaps with researchers, domain experts and customers.
- Contribute to technical reports, benchmarks and research insights.
What they’re looking for
- Background in applied AI research, research engineering, machine learning or LLM-based systems.
- Hands-on experience designing evaluations for AI agents, including tool use, multi-step workflows and task completion.
- Strong Python skills and experience building evaluation pipelines or ML infrastructure.
- Experience developing benchmarks, datasets, metrics and experimental methods for AI systems.
- Ability to analyse agent failures, find root causes and turn findings into measurable gains.
- Understanding of LLM behaviour, agent architectures and challenges in reliability and robustness.
- Ability to turn ambiguous real-world problems into structured experiments.
- Strong collaboration and communication skills across research, engineering and customer teams.
Nice to have
- Experience evaluating or optimising AI agents in production or enterprise settings.
- Familiarity with reinforcement learning, reward modelling or post-training techniques.
- Experience building research infrastructure or automated experimentation frameworks.
- Prior experience at a frontier AI lab or in an applied AI research environment.
- Contributions to research publications, open-source benchmarks or evaluation frameworks.
- Experience working directly with enterprise customers or technical stakeholders.
What’s on offer
- Base salary of $200,000 to $250,000 USD nationally, with the posting also citing a $300,000 to $700,000 per year pay figure.
- Equity compensation for all employees.
- Possible performance-based bonuses.
- Up to 100% reimbursement of health insurance premiums.
- Paid time off and a 401(k) plan with company match.
Skills
Questions about this role
Is this role remote?
Yes, it is fully remote and open to candidates based in 55 countries, including the United States, United Kingdom, Canada, Germany, India, France and the Netherlands.
How much does it pay?
The posting gives a base salary of $200,000 to $250,000 USD, and separately lists a pay figure of $300,000 to $700,000 per year. Equity and possible bonuses are also mentioned.
Is this a full-time role?
Yes, it is listed as full-time.