Get Started
Hunting for you, 24x7Get the iPhone app

Your hunt runs in your pocket, and tells you the moment a strong match opens.

Download on theApp Store
Hippocratic AIPosted 18 days ago

Staff Site Reliability Engineer

Who we'd reach at Hippocratic AIFound and reached by Power Apply
Best leadHiring manager

Leads the team this role joins

Recruiter

In-house, owns the opening

Posted this job

Put the opening up

Power Apply

From your own Gmail and LinkedIn.

About the role

This is a full-time, on-site staff site reliability engineering role in Gurugram, India, for a senior engineer with at least ten years across SRE, DevOps and software engineering. The core project is a GPU management and scheduling platform for nearly 30 AI models running on mixed hardware, including admission control and automatic scaling of model replicas. The engineer works in the office five days a week and needs some overlap with a US-based team in California hours.

What you’ll do

  • Design and build the GPU management and scheduling platform that decides when and where inference calls run.
  • Build the metrics pipeline that collects GPU load and utilization data and turn those signals into decisions.
  • Implement admission control that accepts, queues or sheds inference requests to stay within fleet limits.
  • Build autoscaling that adjusts model replicas to real-time demand and utilization.
  • Develop cloud orchestration systems and operators in Python and Go.
  • Architect scalable, fault-tolerant and secure production systems on AWS, GCP or Azure.
  • Build infrastructure automation and deployment pipelines with Terraform and CI/CD.
  • Set up monitoring, logging and alerting for platform reliability.
  • Define and enforce security and compliance policies for a healthcare AI platform.
  • Work with engineers and research scientists on infrastructure and operational issues, and mentor engineers.

What they’re looking for

  • At least ten years of professional experience across site reliability, DevOps and software engineering.
  • A computer science degree from a top CS program, which the posting lists as required.
  • Strong software engineering fundamentals in Python or Go, including building orchestration and scheduling systems.
  • Experience designing systems that make decisions from operational metrics, such as autoscaling, load shedding or admission control.
  • Deep experience with infrastructure automation and CI/CD, such as Terraform or GitLab CI/CD.
  • Hands-on production experience with AWS, GCP or Azure.
  • Strong knowledge of Docker and Kubernetes.
  • Experience with monitoring and logging stacks such as ELK, Grafana or Datadog.
  • Familiarity with secrets management and security tools such as HashiCorp Vault, AWS KMS or Azure Key Vault.
  • Strong problem solving, communication and interpersonal skills.

Nice to have

  • Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators.
  • Familiarity with ML inference serving such as Triton, KServe or Ray Serve.
  • Experience with Kubernetes autoscaling internals, including HPA, VPA, custom metrics and custom controllers.
  • Experience implementing HIPAA and SOC 2 compliance.
  • Experience operating in an HPC environment.
  • A master's degree in computer science, computer engineering or a related field.

Skills

Questions about this role

Is this role on-site?

Yes. The team is expected to be in the Gurugram office five days a week.

How many years of experience do I need?

The posting asks for at least ten years across SRE, DevOps and software engineering.

How do I apply?

You apply through Hippocratic AI's own site, where you will be taken after clicking the apply link.