Who we'd reach at Technical AdvisorFound and reached by Power Apply
Best leadEngineering manager
Leads the team you would join
Technical recruiter
Owns the opening
Team member
On the team, can refer you
From your own Gmail and LinkedIn.
About the role
This is a senior, remote contract-style evaluation role for experienced engineers with at least eight years of production experience. The work involves evaluating how AI coding agents behave on real codebases, reviewing model-written pull requests and building hard test problems for frontier models. The posting does not name the employer and does not state a location beyond remote. Candidates should have strong written communication skills and be able to explain why code fails.
What you’ll do
- Analyze how AI coding agents behave across full coding sessions.
- Review model-written pull requests against production repositories and document issues with severity and rationale.
- Design and build container-based evaluation benchmarks for frontier models.
- Work alongside AI researchers to write root-cause analyses of model failures.
- Write original technical rationale for every evaluation without AI-generated text.
What they’re looking for
- At least 8 years of production engineering experience, with exceptions for exceptional profiles.
- Background as a senior, staff or principal engineer, tech lead or open-source maintainer.
- Experience in production codebases with a strict code review culture.
- Python and TypeScript experience, with the ability to work in unfamiliar languages.
- Comfort with Docker, git and the command line.
- Cross-layer fluency across backend, frontend, APIs, data, testing or developer tooling.
- Strong written communication skills.
What’s on offer
- Remote work.
Skills
Questions about this role
Is this role remote?
Yes, the posting lists the role as remote.
How many years of experience do I need?
At least 8 years of production engineering experience is preferred.