Leads the team this role joins
In-house, owns the opening
Director or head of the group
From your own Gmail and LinkedIn.
About the role
This is a remote, task-based contract for mid-level software engineers in India who design coding tasks and grade AI model output. Pay is hourly, typically 15-20 hours per week, and some tasks carry a flat rate. The employer, Patronus AI, describes itself as an AI evaluation and research group that works with model developers and large enterprises.
What you’ll do
- Write realistic, multi-step coding and technical tasks that test how a model reasons, writes code and follows precise instructions.
- Build tasks from realistic material such as codebases, specs, architecture documents and datasets.
- Assess AI-generated code and technical writing for correctness, efficiency, clarity and real-world practice.
- Score model outputs against rubrics and reference answers, and write structured feedback.
- Review other contributors' work for accuracy and clarity.
- Work asynchronously and meet deadlines.
What they’re looking for
- At least two years of professional software engineering experience, with internships excluded.
- Experience with system design, data pipelines, or ML or LLM-based applications.
- Strong coding in at least one major language such as Python, Java, C++, JavaScript, TypeScript or Go.
- Experience building, reviewing and debugging production-grade code.
- Excellent written English and close attention to detail.
- A bachelor's degree in computer science or a related field, or equivalent experience.
- All task content must be human-authored, with no LLM-generated code or text and nothing copied from searchable sources.
Nice to have
- Prior work in code review, technical writing, AI training or model evaluation.
What’s on offer
- Pay of $50-$100 per hour, with some tasks paid at a flat rate.
- A flexible commitment of typically 15-20 hours per week, which can run alongside a full-time job.
- Remote work from India on a fixed-term, task-based contract.
- Applications are reviewed on a rolling basis, and placement depends on project needs and assessment results.
Skills
Questions about this role
How much does it pay?
The listing states $50-$100 per hour, and some tasks use a flat rate.
How many hours per week?
Typically 15-20 hours per week, with flexible timing.
How many years of experience are required?
At least two years of professional software engineering experience, with internships excluded.
Is it remote?
Yes. The role is remote and the listing is for candidates in India.