Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
RemoteJobsOnePosted 2 days ago

Member of Technical Staff, Coding Agent Research & Evaluations – Remote (Worldwi

PrincipalRemoteFull-time$600,000 – $1,300,000 a year
Who we'd reach at RemoteJobsOneFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Team lead

Runs the team day to day

Founder

Decides on early hires

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a fully remote, full-time, principal-level role open to candidates in 55 countries, including the Philippines, United States, United Kingdom, Canada, Germany, India, France and the Netherlands. The work centers on designing benchmarks, scoring methods and datasets that measure frontier coding agents, then communicating the results. RemoteJobsOne lists the role for an unnamed employer and asks for at least three years of relevant experience.

What you’ll do

  • Design and own evaluation frameworks for coding agents, including benchmark specs, scoring methods, rubrics and quality standards.
  • Lead research into how coding models perform across varied software engineering tasks.
  • Build datasets, golden examples and evaluation protocols for frontier coding systems.
  • Analyze model behavior and failure modes, then turn findings into changes for training and evaluation.
  • Build tooling for large-scale experiments, data generation, review workflows and evaluation pipelines.
  • Set standards for coding-agent assessment with a focus on rigor, reproducibility and measurement quality.
  • Work with researchers, engineers and applied AI teams to design experiments.
  • Contribute to technical reports, benchmark studies and client-facing research.

What they’re looking for

  • Strong software engineering background in Python, C++ or a comparable language.
  • Three or more years in software engineering, machine learning, AI research, evaluation or a related field.
  • Experience designing, reviewing or validating assessments, benchmarks, coding tasks or evaluation methods.
  • Familiarity with large language models, coding agents, reinforcement learning or model evaluation.
  • Ability to build tooling, automate workflows and improve processes through experiments.
  • Strong analytical skills for investigating model behavior.
  • Excellent written and verbal communication for mixed audiences.
  • Comfort in fast-moving research settings with ambiguity and shifting priorities.

Nice to have

  • Work on frontier AI systems, coding agents or model evaluation research.
  • Experience designing benchmarks or datasets for machine learning at scale.
  • Familiarity with agentic workflows, tool use, reinforcement learning or post-training.
  • Publications, open-source contributions or technical leadership.
  • Track record of driving ambiguous projects from concept to execution.

What’s on offer

  • Full-time, fully remote.
  • Base salary range of 200,000 to 260,000 USD per year, according to the compensation notice.
  • Equity eligibility and possible performance bonuses.
  • Up to 100% reimbursement of health-insurance premiums, paid time off and a 401(k) with company match.
  • The headline pay line on the listing reads 600,000 to 1,300,000 USD per year, which differs from the base range above, so confirm the figure with the employer before applying.

Skills

Questions about this role

Is this role remote?

Yes. It is fully remote and open to candidates in 55 countries.

How much does it pay?

The compensation notice gives a base salary of 200,000 to 260,000 USD per year, plus equity and possible bonuses. The headline pay line shows 600,000 to 1,300,000 USD per year, so confirm the figure with the employer.

How many years of experience do I need?

Three or more years in software engineering, machine learning, AI research, evaluation or a related field.