Senior AI Product Engineer
Skills
About the role
This is a senior product engineering role at Health Note, a healthcare AI company, fully remote within the United States. It suits an engineer who wants to make an AI agent measurably better in production through data, evals and tooling rather than pure research. The role focuses on connecting agent behavior, workflows and outcome data to drive continuous improvement of the company's Access Agent.
What you’ll do
- Own improvements to the Access Agent from investigation through production rollout
- Build ways to measure agent performance including evals, dashboards and logs
- Analyze transfer patterns, containment rates and task success to find improvements
- Build tooling connecting prompts, configurations and outcomes
- Create checks to catch regressions before and after changes go live
- Partner with Product, Operations and Engineering to prioritize improvements
- Design lightweight experiments for model, prompt and workflow changes
- Build dashboards and reporting for agent performance visibility
What they’re looking for
- 5+ years of software engineering experience building production systems
- Experience shipping agentic systems into production and measuring outcomes
- Strong analytical skills translating ambiguous problems into measurable outcomes
- Ability to use metrics, logs and experiments to judge whether changes worked
- Strong product instincts connecting technical decisions to business outcomes
- Demonstrated ability to independently own initiatives end to end
- Experience using AI-assisted engineering tools like Claude, Cursor or Codex
Nice to have
- Experience building eval frameworks for AI products
- Familiarity with Langfuse or other observability platforms
- Experience with RAG, agent orchestration or prompt management
- Experience measuring and improving customer-facing AI experiences
- Healthcare technology or regulated-industry experience
- Experience working in startup environments with ambiguity
Questions about this role
Is this role remote?
Yes, it's remote within the United States.
How many years of experience do I need?
5+ years of software engineering experience building production systems.
What does the Access Agent do?
It's Health Note's AI system handling patient intake, referrals, medication refills and access workflows.
Related roles
Senior AI-ML Engineer
A senior AI/ML engineering role at Ericsson in Bengaluru, building GenAI and machine learning capabilities across products and developer tooling.
senior ai engineer (product neuro forge team)
Remote Senior AI Engineer role covering production AI features, model development, RAG, and semantic search. It suits an experienced Python and ML engineer with deployed NLP systems.
Senior AI-Native Engineer
Senior hybrid AI engineering role in Gurgaon for an engineer with five or more years of experience and production AI/ML background. You'll design agentic applications, evaluation frameworks, and monitored AI capabilities end to end.
IN_Senior Associate_Azure AI/ML/Agentic AI Engineer_GCC_Advisory_Bangalore
An on-site Bengaluru role building agentic AI and RAG applications on the Azure AI stack at PwC's Advisory GCC. Suited to a Python engineer with 4-8 years of experience in Azure AI and agentic frameworks.
IN_Senior Associate_Azure AI/ML/Agentic AI Engineer_GCC_Advisory_Bangalore
An on-site Bengaluru role building agentic AI and RAG applications on the Azure AI stack at PwC's Advisory GCC. Suited to a Python engineer with 4-8 years of experience in Azure AI and agentic frameworks.
Senior Software Engineer (Python, AI ML Engineer)
Optum is hiring a senior software engineer in Gurugram to build and operate AI, ML, and LLM-powered enterprise solutions. The role suits engineers with production AI experience, Python or similar software skills, and knowledge of RAG, orchestration, cloud services, and evaluation.