Who we'd reach at SoftwareRemoteJobsFound and reached by Power Apply
Best leadHiring manager
Leads the team this role joins
Recruiter
In-house, owns the opening
Posted this job
Put the opening up
From your own Gmail and LinkedIn.
About the role
This is a part-time remote role for an experienced AI engineer with 6 to 13 years of experience, hosting and serving foundation models in production. The work centers on making LLM inference fast and scalable across multi-GPU setups on AWS, using Docker and Kubernetes. The posting is marked urgent with one opening, about 2 hours a day across 5 days a week, aligned to India Standard Time.
What you’ll do
- Build and maintain infrastructure for hosting and serving foundation models in production.
- Deploy and tune large language models for efficient inference and scale.
- Apply inference optimizations such as continuous batching, KV caching, PagedAttention and quantization.
- Build distributed inference across multi-GPU setups using tensor and data parallelism.
- Improve serving latency, throughput, GPU utilization and memory use.
- Build scalable AI infrastructure on AWS, Docker and Kubernetes.
- Add monitoring, logging, autoscaling and load balancing for production AI systems.
- Work with researchers and engineers to deliver reliable AI solutions.
- Troubleshoot production issues and improve the reliability of model-serving platforms.
What they’re looking for
- 6 to 13 years of experience.
- Required skills: PyTorch, Python, AI and LLMs.
What’s on offer
- Part-time, about 2 hours a day across 5 days a week.
- Remote work.
Skills
PyTorchPythonLLMsLarge Language ModelsInference OptimizationModel ServingKV CachingQuantizationContinuous BatchingDistributed InferenceTensor ParallelismAWSDockerKubernetesGPU InfrastructureAutoscalingLoad BalancingMonitoring
Questions about this role
Is this role part-time?
Yes, about 2 hours a day, 5 days a week.
How much experience is needed?
6 to 13 years.
Is it remote?
Yes, the role is remote and aligned to India Standard Time.
Which skills are required?
PyTorch, Python, AI and LLMs.