Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
G-PDate not stated

Senior AI Product Manager, Evals & AnnotationsNew

United StatesSeniorRemote$200,000 – $250,000 a year
Who we'd reach at G-PFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Talent partner

Hires for this team

Senior teammate

On the team, can refer you

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a senior remote product manager role in the United States that owns the evaluation and annotation product area at Arize AI. The person hired will shape the full evaluation lifecycle, from creating and running evals to human review and iteration, for teams building AI agents and applications. The role calls for hands-on technical depth, strong product and UX judgment, and frequent customer conversations. The posting notes that Arize has been acquired and that the team still works at an early-stage startup pace.

What you’ll do

  • Own the evals and annotations product area, from eval creation and execution through human review and analysis.
  • Set product strategy and the roadmap, balancing customer needs, platform investment, UX improvements and new AI opportunities.
  • Prototype workflows and work with design to turn them into polished, intuitive experiences.
  • Work with engineers on evaluation execution, backend orchestration and data infrastructure, including technical tradeoffs.
  • Talk with customers several times a week to learn how they evaluate and improve AI systems.
  • Make prioritization decisions and take products from problem definition through launch and adoption.
  • Help shape evaluation direction, including the integration with Dynatrace.

What they’re looking for

  • Significant product management experience owning technically complex, developer-facing products, with flexibility on exact years for exceptional candidates.
  • Working knowledge of LLM evaluations, automated evals, human feedback and experimentation.
  • Experience with developer tools, AI infrastructure, ML platforms or similar technical products, plus a solid grasp of backend systems and architecture.
  • Ability to debug independently, read code, build prototypes and hold credible technical discussions with engineers.
  • Strong product and UX instincts and the ability to prototype core experiences with designers.
  • Ownership, initiative and comfort with ambiguity.
  • Experience working closely with technical customers and strong communication skills.

Nice to have

  • Direct experience with LLM evaluation platforms, annotation systems, human-in-the-loop workflows or agent orchestration.
  • A strong backend or platform engineering background, including distributed systems, workflow orchestration or data infrastructure.
  • Hands-on programming across backend and frontend technologies.
  • Experience with LLM-as-a-judge, custom evaluators, experiment tracking, datasets or feedback-driven improvement.
  • Previous engineering experience or substantial hands-on software development.
  • Early-stage or founder-led startup experience.

What’s on offer

  • Estimated annual salary of $200,000-$250,000 plus a competitive equity package, with actual pay based on job-related factors.
  • Medical, dental and vision coverage, a 401(k) plan, unlimited paid time off, generous parental leave and mental and wellness support.
  • Remote-first work, with optional offices in New York City and the San Francisco Bay Area, and a monthly WFH stipend for co-working spaces for remote staff.

Skills

Questions about this role

Is this role remote?

Yes. The role is listed as remote in the United States, and the company is remote-first with optional offices in New York and the San Francisco Bay Area.

How much does it pay?

The estimated annual salary is $200,000-$250,000, plus an equity package.

How many years of experience do I need?

The posting asks for significant product management experience but is flexible on exact years for candidates with exceptional technical depth and ownership.