Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
MpathicPosted 2 days ago

Machine Learning Engineer, Evaluation

Who we'd reach at MpathicFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Founder

Decides on early hires

Head of talent

Runs hiring at the company

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a full-time, mid-level remote role in the United States for a machine learning engineer who evaluates ML products. It suits engineers with two to five years of experience shipping ML systems who care about measuring quality, safety and reliability. The role is with mpathic, which builds AI that analyzes clinical and human conversations.

What you’ll do

  • Build, deploy and maintain ML models and services alongside engineering and product teams.
  • Design evaluation methods that match product needs and real operating conditions.
  • Build and maintain evaluation datasets, benchmarks and automated regression tests covering safety, empathy and risk.
  • Choose metrics, analyze performance across data segments and investigate failure modes.
  • Design experiments and use statistics to separate real improvements from noise.
  • Connect offline evaluation results to what customers see in production.
  • Build tools that make evaluation repeatable across development and release.
  • Explain findings and limits to clinicians, product leads and customers.

What they’re looking for

  • Two to five years of practical experience shipping ML systems, or equivalent
  • Strong Python and familiarity with common ML frameworks
  • Solid ML fundamentals, including training, validation and generalization
  • Experience with evaluation metrics, benchmark design and systematic error analysis
  • Understanding of statistical inference and experimental design
  • Ability to spot data leakage, evaluation bias and test coverage gaps
  • Good software engineering habits: testing, version control and code review
  • Skill turning vague product goals into measurable quality criteria

Nice to have

  • A/B testing and production performance monitoring
  • Human evaluation and annotation guidelines, including inter-rater agreement
  • Speech or conversational data experience, such as transcripts and diarization
  • Evaluating ranking, retrieval, recommendation or generative AI systems
  • Robustness, calibration, fairness or distribution shift work
  • Building evaluation tooling and quality checks in CI/CD
  • Clear written and verbal communication, and startup or cross-functional team experience

What’s on offer

  • Full-time, remote working
  • The listing marks visa sponsorship as available

Skills

PythonMachine LearningGenerative AIEvaluation MetricsA/B TestingStatisticsExperimental DesignVersion ControlCI/CDRegression TestingBenchmark DesignError AnalysisHuman EvaluationData Leakage DetectionSpeech DataConversational Data

Questions about this role

How many years of experience does this role need?

Two to five years of relevant work, or equivalent.

Is this role remote?

Yes, the role is remote.

Does the employer sponsor visas?

The listing marks visa sponsorship as available.