Software Engineer
Skills
About the role
This is a remote freelance contract for software engineers anywhere in the world who can audit tasks used to train and evaluate AI systems. You will test SWE-Bench tasks for technical accuracy, reproducibility, solvability, and adequate evaluation coverage. The work suits experienced engineers with strong codebase-navigation, debugging, and technical review skills.
What you’ll do
- Assess whether software engineering tasks are realistic, accurate, solvable, and reproducible.
- Review tests and evaluation criteria for reliability.
- Identify codebase integration problems, test failures, and logic errors.
- Write clear, actionable technical feedback.
What they’re looking for
- Professional software engineering experience and deep technical expertise.
- Experience navigating complex codebases and developing real-world applications.
- Familiarity with SWE-Bench standards.
- Strong analytical, troubleshooting, and problem-solving skills.
- Expertise in one relevant software engineering specialty is sufficient.
What’s on offer
- Freelance contract work performed remotely worldwide.
- Pay ranges from $70 to $100 per hour, with the final rate based on experience, expertise, and geographic location.
- Contractors provide a secure computer and high-speed internet.
- Employer-sponsored health insurance and paid time off are not included.
Questions about this role
Is this software engineering auditor role remote?
Yes. The workplace is remote and the location is worldwide.
How much does the contract pay?
The stated range is $70 to $100 per hour, with the final rate determined after evaluating experience, expertise, and location.
What type of work is involved?
You will audit software engineering tasks used to train and evaluate AI systems.
Are benefits included?
No. Contractors supply their own computer and internet, and company-sponsored health insurance and PTO do not apply.
Related roles
Machine Learning (ML) AI Task Auditor - Freelance AI Trainer Project
This remote freelance project needs machine-learning specialists to audit AI training and evaluation tasks. It suits practitioners with hands-on model development, data-pipeline and PyTorch or TensorFlow experience.
Software Development Engineer, Amazon Traffic Engineering
A software development engineer role at Amazon in Hyderabad, building service onboarding and tax-compliance systems at scale.
Forward Deployed Engineer - Software Engineer - North America
A remote-first forward deployed engineer role at ElevenLabs embedded with strategic customers to deliver technical integrations. Suited to a Python-proficient engineer who enjoys customer-facing work and regular travel to client sites.
Senior Software Engineer - DevOps Engineer
A senior DevOps engineering role at Wells Fargo in Bengaluru, building CI/CD pipelines and driving automation.
Senior Software Engineer (Python, AI ML Engineer)
Optum is hiring a senior software engineer in Gurugram to build and operate AI, ML, and LLM-powered enterprise solutions. The role suits engineers with production AI experience, Python or similar software skills, and knowledge of RAG, orchestration, cloud services, and evaluation.
Software Engineering Senior Manager - Data Engineering - GEN AI
A senior engineering management role leading data engineering teams for Wells Fargo's Treasury Technology group in Bengaluru. Suited to an experienced people leader with a data engineering background in regulated financial services.