Leads the team you would join
Owns the opening
On the team, can refer you
From your own Gmail and LinkedIn.
About the role
This is a remote freelance project for an experienced machine learning practitioner to design original benchmark challenges for an agent evaluation platform. It suits ML experts who can build self-contained, reproducible tasks with clean code, documentation, and reference solutions. Each accepted challenge includes a commercially usable dataset, deterministic grading scripts, a README, and a reference notebook. Applicants are asked to send a link to past work.
What you’ll do
- Build or source a commercially usable dataset, either public domain, permissively licensed, or self-generated.
- Write deterministic prepare and grade scripts that run headless on Linux in a single command under Python 3.10.
- Write a README covering objectives, data schema, evaluation metric, and submission format.
- Expose at least five task-specific rubric checks as JSON from the grading script.
- Provide a documented reference solution in a Jupyter notebook that reaches a reasonable benchmark score.
- Keep the work reproducible and free of copyrighted or LLM-generated filler material.
What they’re looking for
- Machine learning expertise, shown through well-built notebooks or GitHub projects.
- Comfort with Python 3.10 and pip-installable repositories.
- Experience with Pandas, scikit-learn, XGBoost, LightGBM, PyTorch, or TensorFlow, chosen to suit each task.
- Ability to make repositories run end to end on GitHub-hosted continuous integration.
- Skill in designing Kaggle-style problems.
What’s on offer
- USD 200 per accepted task, paid once the repository clears a code and licensing review.
- The listing also shows a budget range of 30 to 250 USD.
Skills
Questions about this role
How much does it pay?
The posting says USD 200 per accepted task after a code and licensing review. It also lists a budget range of 30 to 250 USD.
Which Python version is required?
Python 3.10, with scripts that run headless on Linux.
Is this role remote?
Yes, the listing is remote.