Senior Data Scientist, AI Scoring & Evaluation
Skills
About the role
This is a senior data science role at Workera focused on the accuracy, fairness and quality of the company's AI-driven skills assessment scoring. It suits a data scientist who has independently driven production data science work and wants to own an evaluation system end to end, including how the underlying AI is prompted and orchestrated. The role sits at the intersection of engineering, assessment science and design, reporting to the VP of Product and Assessments.
What you’ll do
- Own scoring quality end to end: evaluator design, rubric anchoring, calibration and accuracy against expert benchmarks
- Build and run a continuous evaluation harness with gold sets, bias diagnostics, drift detection and a pre-release quality gate
- Define and publish assessment quality KPIs on company-wide dashboards
- Lead analytics investigations behind assessment decisions, including root cause analysis when scores look wrong
- Implement and coordinate measurement improvements end to end, owning the spec and validation
- Act as liaison between AI governance and InfoSec on frameworks like GDPR and the EU AI Act
- Build AI tooling and documentation so partner teams can get data and analysis autonomously
What they’re looking for
- 4+ years of production data science work (or 3+ years with demonstrated end-to-end ownership), ideally at an AI startup or fast-moving product environment
- Strong Python and SQL skills, able to run analyses independently
- Real statistical and analytical depth, able to design defensible studies without being handed the design
- Experience with quantitative educational, learning or assessment data, or strong appetite to go deep on it
- Experience working with LLM-based systems in production, including prompt design and evaluating model output against human judgment
- Ability to explain quantitative results to non-technical audiences including executives and customers
Nice to have
- Background in psychometrics or educational measurement
Questions about this role
How many years of experience do I need?
The posting asks for 4+ years of production data science work, or 3+ years with demonstrated end-to-end ownership.
Do I need a background in psychometrics?
No, it's called a plus rather than a requirement — assessment scientists coach you on it.
Is this a data science or an engineering role?
It's a data science role embedded with the assessment tech and assessment science teams, partnering closely with product engineers.
Related roles
Senior Data Scientist / Lead Data Scientist
A senior/lead data scientist role at Celebal Technologies across Noida, Jaipur or Bengaluru, leading enterprise AI/ML and generative AI delivery.
Data Scientist-Vice President-Finance Data & Analytics
A Vice President-level data science role at JPMorgan Chase in Columbus, Ohio, focused on financial analytics and AI tooling. Suits an experienced financial analytics professional skilled in SQL, cloud platforms and Databricks.
Data Analyst - Data Quality & Databricks
A data analyst role centered on data quality, profiling and reconciliation using Databricks, SQL and PySpark, open across Bangalore, Chennai and Pune for candidates with 7-11 years of experience.
Senior Data Scientist
A senior data scientist role at Optum (UnitedHealth Group) in Hyderabad, building ML models and MLOps pipelines on Azure and Databricks.
Senior Data Engineer, Data Platform & ML (Remote)
A remote senior data engineering role in Yerevan focused on feature-store development, data services, streaming, and production data pipelines. It is suited to an engineer with scalable systems experience across data, machine learning, or backend development.
Senior Data Scientist - ML, Python
A senior data scientist role at HCL Technologies across several Indian cities, focused on machine learning and NLP work in Python. Suited to an experienced data scientist strong in TensorFlow, PyTorch and large-scale data analysis.