Leads the team you would join
Signs off on engineering hires
Owns the opening
From your own Gmail and LinkedIn.
How this role compares
One of 38 open Senior Backend Engineer roles we are tracking today.
This one advertises $220,000 – $260,000 a year. The middle of the 8 that state a figure advertises $80,000 – $120,000 a year.
Posted 2 days ago, with 4 of the others posted since and 33 still open from before.
Counted across the roles Frontrunner has open right now.
About the role
This is a full-time, onsite senior backend engineering role in San Francisco, for engineers with at least two years of production backend experience. The work centres on the core of LiteLLM's AI gateway, covering routing, authentication, caching, spend tracking, guardrails and observability for very high request volumes. LiteLLM is a team of about twenty that maintains an open-source Python SDK and proxy for calling more than 100 LLM APIs.
What you’ll do
- Own core proxy components end to end, including design, production and incidents.
- Find and remove whole classes of bugs, such as race conditions, stale cache state across replicas and silently swallowed errors.
- Make guardrail and policy checks logged, traceable and loud on failure.
- Build in-gateway guardrails that block common attacks on LLM APIs, MCP servers and agents.
- Keep observability integrations accurate at over one billion requests per month, with low latency overhead and no memory leaks.
- Help move performance-critical paths from Python to Rust without changing behaviour.
- Raise the team's standards for code review, testing and debugging.
What they’re looking for
- At least two years building production backend systems, with real ownership of mission-critical software.
- Strong Python and experience with async or high-throughput services such as asyncio or FastAPI.
- A record of improving production reliability through refactors, re-architectures, migrations or root-cause fixes.
- Strong debugging across services, databases such as Postgres and Redis, and distributed systems.
- High ownership of the customer impact of what you ship.
- A BS or MS in computer science or a related field, or equivalent proof of exceptional ability.
Nice to have
- Rust, Go or C++ in production.
- Time at a fast-growing startup with a large, real codebase.
- Experience with LLM APIs, proxies or API gateways.
- Open-source contributions.
What’s on offer
- Base salary of USD 220K-260K per year, plus equity.
- Health, dental and vision coverage.
- Onsite work in San Francisco.
Skills
Questions about this role
How much does it pay?
The base salary is USD 220K-260K per year, plus equity.
Is this role remote?
No, the role is onsite in San Francisco.
Do I need a degree?
A BS or MS in computer science or a related field is listed, or equivalent proof of exceptional ability.