Leads the team this role joins
Decides on early hires
Runs hiring at the company
From your own Gmail and LinkedIn.
About the role
This is a full-time, fully remote role based in India, offered through Jobgether on behalf of a partner company. The specialist keeps production AI infrastructure reliable, secure, and cost-efficient, working on LLM inference, model serving, vector databases, and RAG platforms. The role sits between site reliability engineering, distributed systems, and machine learning infrastructure, and suits engineers with several years of production operations experience.
What you’ll do
- Operate, scale, and tune multi-tenant LLM inference, model-serving, vector database, RAG, and MCP platforms across cloud and GPU clusters.
- Set SLOs and SLIs and build health checks and observability dashboards.
- Automate deployment rollouts, autoscaling, canary releases, self-healing, and disaster recovery.
- Improve GPU utilisation and inference throughput, latency, and batching.
- Lead production incident responses and root-cause investigations, and put lasting fixes in place.
- Run production readiness reviews with AI researchers and platform teams.
- Take part in a sustainable on-call rotation.
What they’re looking for
- A Bachelor's degree in a related field with at least 5 years of relevant experience, or a Master's with at least 3 years.
- At least 3 years of software development in Python, Go, or Java.
- Experience running distributed, containerised production systems on Kubernetes with CI/CD.
- Background in site reliability engineering, production operations, and distributed systems troubleshooting.
- Practical experience with Prometheus, Grafana, Splunk, or OpenTelemetry.
- Working knowledge of AWS, Azure, or Google Cloud.
Nice to have
- Hands-on experience with vLLM, TensorRT-LLM, Ray Serve, or KServe.
- Familiarity with GPU orchestration, capacity planning, and performance profiling.
- Experience with Milvus, Pinecone, or Redis.
- Understanding of RAG pipelines, model routing, MCP, and agent runtimes.
- Experience with multi-region high-availability design, automated remediation, or chaos engineering.
What’s on offer
- Fully remote working arrangement.
- Full-time employment.
- Exposure to distributed systems, GPU computing, and modern ML platforms.
- Opportunities for technical learning and professional development.
Skills
Questions about this role
Is this role remote?
Yes, it is a fully remote arrangement, and the role is based in India.
How much experience do I need?
At least 5 years of relevant experience with a Bachelor's degree, or at least 3 years with a Master's, plus 3 years of software development in Python, Go, or Java.
Does the posting state a salary?
No, the posting does not state a salary.