Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
RemoteJobsOneDate not stated

Member of Technical Staff, Coding Agent Research & Evaluations

Remote in a region — 6 countriesMid levelRemoteFull-time$200,000 – $260,000 a year
Who we'd reach at RemoteJobsOneFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Recruiter

In-house, owns the opening

Posted this job

Put the opening up

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a full-time, fully remote role for a mid-level engineer who evaluates and improves frontier coding agents. It suits engineers who combine software engineering, machine learning and model evaluation skills, and who are comfortable in fast-moving research settings. The company is remote-first and states that the role is open to candidates in a list of countries that includes the United States, United Kingdom, Canada, Germany, India and the Philippines.

What you’ll do

  • Design and own evaluation frameworks for coding agents, including benchmark specs, scoring methods, rubrics and quality standards.
  • Lead research on measuring and improving coding model performance across software engineering tasks.
  • Build datasets, golden examples and evaluation protocols for frontier coding systems.
  • Analyze model behavior and failure modes, and turn findings into training and evaluation improvements.
  • Build tooling for large-scale experiments, data generation, review workflows and evaluation pipelines.
  • Set best practices for coding-agent assessment with a focus on rigor and reproducibility.
  • Work with researchers, engineers and applied AI teams to design experiments.
  • Contribute to technical reports, benchmark studies and research communication.

What they’re looking for

  • Strong software engineering background in Python, C++ or a comparable language.
  • 3 years or more in software engineering, machine learning, AI research, evaluation or a related field.
  • Experience designing, reviewing or validating assessments, benchmarks, coding tasks or evaluation methods.
  • Familiarity with LLMs, coding agents, reinforcement learning or model evaluation.
  • Ability to build tooling, automate workflows and run systematic experiments.
  • Strong analytical skills for investigating model behavior.
  • Clear written and verbal communication for technical and non-technical audiences.
  • Comfort working in fast-moving environments with ambiguity and shifting priorities.

Nice to have

  • Experience working on frontier AI systems, coding agents or model evaluation research.
  • Experience designing benchmarks or datasets for machine learning systems at scale.
  • Familiarity with agentic workflows, tool use, reinforcement learning or post-training methods.
  • Publications, open-source contributions or technical leadership in AI, machine learning or software engineering.
  • A track record of driving ambiguous projects from concept to delivery.

What’s on offer

  • Base salary of $200,000 to $260,000 USD per year.
  • Equity compensation for all employees.
  • Possible performance-based bonuses, subject to role and company policy.
  • Up to 100% reimbursement of health insurance premiums.
  • Paid time off and a 401(k) plan with company match.
  • Fully remote working arrangement.

Skills

Questions about this role

How much does it pay?

The listed base salary is $200,000 to $260,000 USD per year, with equity and possible performance bonuses.

Is this role remote?

Yes. It is fully remote and open to candidates in a list of countries that includes the United States, United Kingdom, Canada, Germany, India and the Philippines.

How many years of experience do I need?

The listing asks for 3 or more years in software engineering, machine learning, AI research, evaluation or a related field.

Is this a full-time job?

Yes, it is a full-time position.