Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
FreelancerPosted 13 days ago

Design Original ML Benchmark Challenges (ML Experts ONLY)

Who we'd reach at FreelancerFound and reached by Power Apply
Best leadEngineering manager

Leads the team you would join

Technical recruiter

Owns the opening

Team member

On the team, can refer you

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a remote freelance project for an experienced machine learning practitioner to design original benchmark challenges for an agent evaluation platform. It suits ML experts who can build self-contained, reproducible tasks with clean code, documentation, and reference solutions. Each accepted challenge includes a commercially usable dataset, deterministic grading scripts, a README, and a reference notebook. Applicants are asked to send a link to past work.

What you’ll do

  • Build or source a commercially usable dataset, either public domain, permissively licensed, or self-generated.
  • Write deterministic prepare and grade scripts that run headless on Linux in a single command under Python 3.10.
  • Write a README covering objectives, data schema, evaluation metric, and submission format.
  • Expose at least five task-specific rubric checks as JSON from the grading script.
  • Provide a documented reference solution in a Jupyter notebook that reaches a reasonable benchmark score.
  • Keep the work reproducible and free of copyrighted or LLM-generated filler material.

What they’re looking for

  • Machine learning expertise, shown through well-built notebooks or GitHub projects.
  • Comfort with Python 3.10 and pip-installable repositories.
  • Experience with Pandas, scikit-learn, XGBoost, LightGBM, PyTorch, or TensorFlow, chosen to suit each task.
  • Ability to make repositories run end to end on GitHub-hosted continuous integration.
  • Skill in designing Kaggle-style problems.

What’s on offer

  • USD 200 per accepted task, paid once the repository clears a code and licensing review.
  • The listing also shows a budget range of 30 to 250 USD.

Skills

Questions about this role

How much does it pay?

The posting says USD 200 per accepted task after a code and licensing review. It also lists a budget range of 30 to 250 USD.

Which Python version is required?

Python 3.10, with scripts that run headless on Linux.

Is this role remote?

Yes, the listing is remote.