Who we'd reach at OpenTrain AIFound and reached by Power Apply
Best leadHiring manager
Leads the team this role joins
Team director
Heads the group this role sits in
Recruiter
In-house, owns the opening
From your own Gmail and LinkedIn.
About the role
This is a remote, part-time contract role for a researcher who designs evaluations for coding agents. It suits engineers who can build benchmarks and scoring methods and explain measurement results clearly. The work is open to applicants across 38 listed countries.
What you’ll do
- Design evaluation frameworks for coding agents, covering benchmark specs, scoring methods, rubrics and quality standards.
- Lead research that measures coding model performance and studies their behavior and failure modes.
- Build datasets and golden examples used to evaluate coding agents.
- Turn evaluation findings into changes for model training and assessment.
- Build tools and infrastructure for experiments, data generation, review workflows and evaluation pipelines.
- Set up repeatable assessment practices for coding agents.
- Share results through technical reports, benchmark studies and research initiatives.
What they’re looking for
- Professional fluency in English.
- Strong software engineering experience in Python, C++ or a comparable language.
- Strong analytical, automation and technical communication skills.
- Experience designing, reviewing or validating technical assessments, benchmarks, coding tasks or evaluation methods.
Nice to have
- Work on frontier AI systems, agentic workflows, tool use or post-training methods.
- Experience with machine learning benchmarks or large-scale datasets.
- Publications, open-source contributions or technical leadership in AI, machine learning or software engineering.
What’s on offer
- Contract, part-time engagement.
- Remote work, open to applicants in the listed countries.
Skills
Questions about this role
Is this role remote?
Yes. It is remote and open to applicants in a list of countries that includes Argentina, Bangladesh, Brazil, India, Mexico and Spain.
Is this a full-time job?
No, it is a part-time contract role.
What experience do I need?
Strong software engineering skills in Python or C++, plus experience designing, reviewing or validating assessments, benchmarks or evaluation methods.