Leads the team this role joins
Decides on early hires
Runs hiring at the company
From your own Gmail and LinkedIn.
About the role
This is a full-time AI SDET role based in India, listed through Jobgether on behalf of a partner company that manages applications and next steps. The role tests and evaluates large language models, AI agents, copilots and intelligent workflows, and builds the automation that supports them. It sits alongside Engineering, AI/ML, Product, Platform and DevOps teams in a globally distributed setting.
What you’ll do
- Design and tune test strategies for AI applications, copilots, chatbots and RAG pipelines.
- Build AI evaluation frameworks that check hallucination, groundedness, toxicity, bias and safety.
- Create automated validation for LLM outputs using assertions, semantic scoring and embeddings.
- Build synthetic and benchmark datasets for AI testing.
- Test multi-agent workflows, tool calling and Model Context Protocol integrations.
- Measure latency, token use, cost and throughput of AI systems.
- Develop UI and API automation in Python, using Robot Framework, Playwright, Selenium or Cypress.
- Test REST, GraphQL, event-driven and microservice architectures.
- Integrate automated tests into CI/CD pipelines with quality gates.
- Validate vector search and retrieval quality in RAG systems.
What they’re looking for
- Strong Python skills for building automation frameworks.
- Solid grounding in testing principles and modern development practice.
- Hands-on work with AI and LLM testing, prompt engineering, evaluation frameworks, RAG and AI agents.
- Experience with CI/CD, Docker, Kubernetes and a major cloud platform.
- Strong API, microservice and UI test automation experience.
- Experience with SQL and data validation.
- Strong debugging and analytical skills.
- Familiarity with Agile and Scrum.
Nice to have
- Experience with Robot Framework and Playwright.
- Knowledge of LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel and MCP.
- Use of AI coding assistants such as Cursor, GitHub Copilot, Claude or Gemini.
- Familiarity with vector databases and semantic search.
- Exposure to MLOps, performance testing or security testing.
- A portfolio of GitHub projects, open-source work or technical writing on AI and testing.
What’s on offer
- Flexible arrangements, with remote, hybrid or office-based options depending on local rules.
- Healthcare coverage, subject to local eligibility.
- Monthly salary reviews and an annual bonus programme.
- Extra paid days off.
- Tuition reimbursement, LinkedIn Learning access and in-company English lessons.
- Internet expense reimbursement.
- An employee referral bonus.
- Mentoring, career progression and more than 20 affinity groups.
Skills
Questions about this role
Is this role remote?
The posting offers remote, hybrid or office-based working, depending on the applicable arrangements.
Do I need a degree?
A bachelor's or master's in a related field is preferred, but equivalent practical experience and demonstrable projects are also valued.
Who is the employer?
The role is listed on behalf of a partner company. Jobgether manages the screening and the hiring company manages next steps.
What benefits are included?
The posting lists healthcare coverage, bonuses, extra days off, learning support and internet reimbursement.