Leads the team this role joins
In-house, owns the opening
Put the opening up
From your own Gmail and LinkedIn.
About the role
This is a mid-level, full-time LLMOps engineering role open remotely or in hybrid form across India, with the team based in Pune. It suits backend and DevOps engineers with four or more years of experience who have run LLM systems in production and can debug non-deterministic behavior. The team sits within Coursera and Udemy, now one combined company, and builds the operational tooling that keeps AI systems reliable for internal teams and enterprise customers.
What you’ll do
- Build and maintain prompt and pipeline versioning, evaluation harnesses and CI/CD workflows designed for non-deterministic AI output.
- Set up monitoring and observability for LLM systems, tracking latency, token use, cost per request, output quality and drift.
- Design and run automated evaluations that catch regressions, hallucinations and quality drops before they reach customers.
- Manage RAG pipelines and vector store infrastructure so retrieval sources stay fresh, accurate and fast.
- Add safety and compliance guardrails, such as content filtering, PII redaction and access controls, in line with privacy and residency rules.
- Control LLM spend through caching, model routing and usage reporting.
- Work with product managers and senior engineers to scope operational needs and turn them into reliable systems.
- Take part in sprint planning, technical grooming and retrospectives.
What they’re looking for
- A bachelor's or master's degree in computer science, computer engineering or a related field.
- 4 or more years of software engineering experience with solid backend and DevOps foundations.
- Hands-on experience running or supporting LLM systems in production, through APIs, RAG pipelines or fine-tuned models.
- Proficiency in a backend language such as Python, Java, TypeScript or Go, plus Docker and Kubernetes.
- Experience building APIs, working with SQL and NoSQL databases and writing automated tests.
- Working knowledge of AWS, GCP or Azure, covering managed and self-hosted services.
- Familiarity with infrastructure as code such as Terraform and CI/CD tooling such as GitHub Actions or Jenkins.
- Comfort debugging production issues in non-deterministic systems, with attention to detail and a bias toward reliability.
Nice to have
- LLMOps tooling such as LangFuse, LangSmith or evaluation frameworks in the style of Weights and Biases.
- Vector databases such as Pinecone, Weaviate or pgvector.
- RAG architectures, MCP implementation and governance, and agent frameworks such as LangGraph.
- LLM gateways such as LiteLLM or OpenRouter, and enterprise AI platforms such as Vertex AI or Bedrock.
- Prompt management treated as code, model access management, deterministic guardrails and evals.
- Understanding of AI governance, including data privacy, residency and compliance.
- Comfort in a fast-paced, ambiguous environment with an ownership mindset.
- Strong communication across engineering, product and strategy teams.
What’s on offer
- Remote or hybrid working across India.
- A team culture described as supportive, innovative and focused on continuous learning.
Skills
Questions about this role
Can I work remotely?
Yes, the role is listed as remote or hybrid across India.
How much experience is needed?
At least four years in software engineering, with backend and DevOps experience.
Is a degree required?
A bachelor's or master's degree in computer science, computer engineering or a related field is listed.