Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
OpenTrain AIPosted 5 days ago

Member Technical Staff Coding Agent Research Evaluation

Who we'd reach at OpenTrain AIFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Team director

Heads the group this role sits in

Recruiter

In-house, owns the opening

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a remote, part-time contract role for a researcher who designs evaluations for coding agents. It suits engineers who can build benchmarks and scoring methods and explain measurement results clearly. The work is open to applicants across 38 listed countries.

What you’ll do

  • Design evaluation frameworks for coding agents, covering benchmark specs, scoring methods, rubrics and quality standards.
  • Lead research that measures coding model performance and studies their behavior and failure modes.
  • Build datasets and golden examples used to evaluate coding agents.
  • Turn evaluation findings into changes for model training and assessment.
  • Build tools and infrastructure for experiments, data generation, review workflows and evaluation pipelines.
  • Set up repeatable assessment practices for coding agents.
  • Share results through technical reports, benchmark studies and research initiatives.

What they’re looking for

  • Professional fluency in English.
  • Strong software engineering experience in Python, C++ or a comparable language.
  • Strong analytical, automation and technical communication skills.
  • Experience designing, reviewing or validating technical assessments, benchmarks, coding tasks or evaluation methods.

Nice to have

  • Work on frontier AI systems, agentic workflows, tool use or post-training methods.
  • Experience with machine learning benchmarks or large-scale datasets.
  • Publications, open-source contributions or technical leadership in AI, machine learning or software engineering.

What’s on offer

  • Contract, part-time engagement.
  • Remote work, open to applicants in the listed countries.

Skills

Questions about this role

Is this role remote?

Yes. It is remote and open to applicants in a list of countries that includes Argentina, Bangladesh, Brazil, India, Mexico and Spain.

Is this a full-time job?

No, it is a part-time contract role.

What experience do I need?

Strong software engineering skills in Python or C++, plus experience designing, reviewing or validating assessments, benchmarks or evaluation methods.