Leads the team this role joins
Decides on early hires
Runs hiring at the company
From your own Gmail and LinkedIn.
About the role
This is a fully remote, part-time-style evaluation role open to candidates worldwide, judging the quality of AI model responses for a research lab. The work involves checking model outputs for factual accuracy, logic, safety and helpfulness, then writing short justifications for each score. The lab uses these evaluations to train large language models for developers and enterprises. It suits candidates with strong written English, analytical reasoning and a degree in a relevant field.
What you’ll do
- Evaluate AI model completions for truthfulness, logical coherence, helpfulness and safety.
- Fact-check model claims against authoritative sources and verified documentation.
- Write justifications that explain reasoning flaws, hallucinations and formatting strengths.
- Classify prompts and outputs into standard taxonomies according to safety guidelines.
- Take part in calibration cycles with senior linguists and research scientists.
What they’re looking for
- A Bachelor's, Master's or PhD in computer science, linguistics, English, philosophy, mathematics or the sciences.
- Strong written English, critical reasoning and editorial judgment.
- Close attention to detail, including spotting subtle logical fallacies in long text.
- Reliable high-speed computer and internet connection.
Nice to have
- Prior experience with LLM prompting, academic research or technical writing.
What’s on offer
- Fully remote work, open to candidates globally.
- Flexible weekly hourly commitments.
- Competitive compensation.
- Opportunities to move into specialized RLHF domain lead roles.
Skills
Questions about this role
Is this AI Prompt Evaluator role remote?
Yes. The role is 100% remote and open to candidates globally.
Do I need a degree?
Yes. A Bachelor's, Master's or PhD in a listed field is required.
What are the working hours?
The posting describes flexible weekly hourly commitments without specifying a fixed schedule.