Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
Patronus AIPosted 18 days ago

Expert: Software Engineer – AI Evaluation (Task-based Contract, Remote)

YemenIraqSaudi ArabiaIranSyriaArmeniaMid levelRemoteContract$50 – $100 an hour
Who we'd reach at Patronus AIFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Recruiter

In-house, owns the opening

Head of the team

Director or head of the group

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a remote, task-based contract for mid-level software engineers in India who design coding tasks and grade AI model output. Pay is hourly, typically 15-20 hours per week, and some tasks carry a flat rate. The employer, Patronus AI, describes itself as an AI evaluation and research group that works with model developers and large enterprises.

What you’ll do

  • Write realistic, multi-step coding and technical tasks that test how a model reasons, writes code and follows precise instructions.
  • Build tasks from realistic material such as codebases, specs, architecture documents and datasets.
  • Assess AI-generated code and technical writing for correctness, efficiency, clarity and real-world practice.
  • Score model outputs against rubrics and reference answers, and write structured feedback.
  • Review other contributors' work for accuracy and clarity.
  • Work asynchronously and meet deadlines.

What they’re looking for

  • At least two years of professional software engineering experience, with internships excluded.
  • Experience with system design, data pipelines, or ML or LLM-based applications.
  • Strong coding in at least one major language such as Python, Java, C++, JavaScript, TypeScript or Go.
  • Experience building, reviewing and debugging production-grade code.
  • Excellent written English and close attention to detail.
  • A bachelor's degree in computer science or a related field, or equivalent experience.
  • All task content must be human-authored, with no LLM-generated code or text and nothing copied from searchable sources.

Nice to have

  • Prior work in code review, technical writing, AI training or model evaluation.

What’s on offer

  • Pay of $50-$100 per hour, with some tasks paid at a flat rate.
  • A flexible commitment of typically 15-20 hours per week, which can run alongside a full-time job.
  • Remote work from India on a fixed-term, task-based contract.
  • Applications are reviewed on a rolling basis, and placement depends on project needs and assessment results.

Skills

Questions about this role

How much does it pay?

The listing states $50-$100 per hour, and some tasks use a flat rate.

How many hours per week?

Typically 15-20 hours per week, with flexible timing.

How many years of experience are required?

At least two years of professional software engineering experience, with internships excluded.

Is it remote?

Yes. The role is remote and the listing is for candidates in India.