Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
OpenTrain AIPosted 4 days ago

Member of Technical Staff, Coding Research (English)

Who we'd reach at OpenTrain AIFound and reached by Power Apply
Best leadTeam lead

Runs the team day to day

Recruiter

In-house, owns the opening

Recent hire

Joined the team recently

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a remote, part-time contract role for engineers with at least three years of experience who design and run evaluations for coding agents, open to applicants in 52 countries. Pay is USD 96.15-125 per hour, and applicants apply through OpenTrain. The work covers evaluation frameworks, benchmark creation and the study of systematic weaknesses in coding systems.

What you’ll do

  • Design evaluation frameworks for coding agents, covering benchmark specifications, scoring methods, rubrics and quality standards.
  • Lead research into coding-model performance and its systematic weaknesses.
  • Develop datasets, golden examples and evaluation protocols.
  • Build experimentation tools, data generation tools, review workflows and evaluation pipelines.
  • Analyse model behaviour and failures, and turn them into improvements for training and evaluation.
  • Establish reproducible practices for coding-agent assessment and report results through technical reports and benchmark studies.

What they’re looking for

  • Fluent English.
  • At least three years in software engineering, machine learning, AI research, evaluation or a related technical field.
  • Strong software engineering with Python, C++ or a comparable language.
  • Experience designing, reviewing or validating assessments, coding tasks, benchmarks or evaluation methods.
  • Familiarity with LLMs, coding agents, reinforcement learning or model evaluation.
  • Ability to build tooling, automate workflows, analyse complex systems and explain findings clearly.

Nice to have

  • Publications, open-source contributions or demonstrated technical leadership in AI, machine learning or software engineering.
  • Experience with frontier AI systems, coding-agent research or agentic workflows.
  • Benchmark or dataset design at scale, tool use, reinforcement learning or post-training methods.

What’s on offer

  • USD 96.15-125 per hour.
  • Part-time contract, fully remote.

Skills

Questions about this role

How many years of experience are needed?

At least three years in software engineering, machine learning, AI research, evaluation or a related discipline.

How much does it pay?

USD 96.15-125 per hour.

Which programming languages are expected?

Python, C++ or a comparable language, along with fluent English.