Leads the team this role joins
In-house, owns the opening
Put the opening up
From your own Gmail and LinkedIn.
About the role
This is a remote, part-time contractor role for senior software engineers with C++ skills who evaluate language models on real software tasks. It is open to applicants in India, Mexico, Bangladesh, Egypt, Ghana, Kenya, Nigeria, Pakistan and Türkiye. The work uses open-source repositories, GitHub issues and test suites, so models are judged on realistic code rather than toy problems. Applicants register on OpenTrain, which then sends them to the hiring platform to complete the application.
What you’ll do
- Triage GitHub issues across open-source C++ and other libraries.
- Set up repositories and development environments, including Docker-based builds.
- Build, modify and test C++ code locally.
- Review unit test coverage and overall software quality.
- Measure how well language models fix bugs in these settings.
- Identify repositories and issues that are hard for models, and support junior engineers on project tasks.
What they’re looking for
- At least three years of software engineering experience.
- Strong C++ skills and the ability to navigate complex codebases.
- Working knowledge of Git, Docker and basic development pipeline setup.
- Comfort running, changing and testing real-world projects locally.
- Fluent English.
Nice to have
- Experience contributing to or evaluating open-source projects.
- Experience in language model research or evaluation.
- Experience building or testing developer tools or automation agents.
What’s on offer
- A remote contractor assignment lasting about three months.
- At least 20 hours per week, including four hours of overlap with Pacific Time.
- The listing does not state a pay rate.
Skills
Questions about this role
Is this role remote?
Yes, the assignment is fully remote.
How many hours per week?
At least 20 hours per week, including four hours of overlap with Pacific Time.
How much does it pay?
The listing does not provide pay details.
How many years of experience are needed?
At least three years of software engineering experience.