Leads the team you would join
Leads the work day to day
Owns the opening
From your own Gmail and LinkedIn.
About the role
This is a remote, part-time contractor role for expert software engineers to write coding tasks and review AI-generated code, open to candidates in 52 countries. The work involves creating reference solutions, tests and evaluation standards that help measure AI systems on software engineering problems. Pay is set per accepted task, equivalent to $60 to $120 per hour, and requires English fluency and a short AI interview.
What you’ll do
- Write demanding software engineering tasks for AI model evaluation.
- Define objective standards for correct solutions and write clear task instructions.
- Build reproducible environments, reference solutions, tests and evaluation criteria.
- Review generated code and technical deliverables for correctness, robustness, completeness and reproducibility.
- Identify problems in implementation, debugging, dependencies, performance, testing and environment setup.
What they’re looking for
- Expert-level software engineering ability.
- Prior paid experience training, evaluating, benchmarking or improving AI systems.
- Strong practical Python and professional experience in another language such as Java, JavaScript, TypeScript, Rust, C++, C# or Go.
- Knowledge of software design, debugging, testing, version control and dependency management.
- Fluent English.
Nice to have
- Substantial repositories, open source contributions, developer tools, publications or benchmarks.
- Experience with AI training tasks, graders or benchmark problems.
- Experience with Linux, Docker, CI/CD, command-line tools or sandboxed environments.
- Experience in programming competitions.
What’s on offer
- Pay of $60 to $120 per hour equivalent, paid per accepted task that meets specifications.
- Fully remote, part-time work of fewer than 20 hours per week.
- Work open to candidates in the listed countries.
Skills
Questions about this role
How much does this role pay?
It pays $60 to $120 per hour equivalent, paid per task that meets the project specifications.
Is this role remote?
Yes. It is fully remote and open to candidates in the listed countries.
How many hours per week?
The role is part-time, with less than 20 hours per week.
How do I apply?
You apply on the hiring platform in about two minutes, then complete a short AI interview, which includes a coding challenge for technical roles.