ML Infrastructure Engineer Jobs

View detail

Deploy, maintain, and optimize production and research compute clusters. Design and implement scalable and efficient ML inference solutions. Develop dynamic and heterogeneous compute solutions for balancing research and production needs. Contribute to productizing model APIs for external use. Develop infrastructure observability and monitoring solutions.

View detail

The responsibilities include architecting and managing horizontally scalable systems to accelerate the end-to-end training lifecycle for Speech-to-Text (STT) and Text-to-Speech (TTS) models, focusing on optimized data preparation, high-throughput training pipelines, distributed infrastructure, and automated evaluation tooling. The role also involves designing and implementing internal UIs and tools to make ML systems and workflows accessible and transparent to non-technical stakeholders. Additionally, the position requires overseeing and managing training tooling, job orchestration, experiment tracking, and data storage.

View detail

The role involves bridging the gap between research and production by transforming cutting-edge algorithms into scalable training systems. Responsibilities include designing and optimizing large-scale training loops and data pipelines, implementing state-of-the-art techniques ensuring numerical stability and computational efficiency, building internal tooling for launching, monitoring, and reproducing complex experiments, diagnosing deep bottlenecks across the training stack such as GPU memory issues, communication overhead, and dataloader stalls, and translating research prototypes into reusable, production-grade infrastructure. The engineer will architect and optimize the core training infrastructure including RL training loops, distributed GPU systems, and large-scale data pipelines, working closely with researchers to build reliable, scalable systems.

View detail

The Senior Engineering Manager, ML Platform at Zoox is responsible for developing and executing a strategic vision for the ML training platform to ensure scalability, reliability, and performance for large-scale Foundation and RL models. They lead the design, implementation, and operation of a robust and efficient ML training platform supporting training, experimentation, validation, and monitoring of ML models. They attract, hire, and inspire a diverse world-class engineering team, fostering a culture of innovation, collaboration, and excellence. The role involves close collaboration with cross-functional teams including ML researchers, software engineers, data engineers, and hardware engineers to define requirements and align architectural decisions. The manager also mentors engineers, providing opportunities for career growth through clear and timely feedback.

View detail

Advance inference efficiency end-to-end by designing and prototyping algorithms, architectures, and scheduling strategies for low-latency, high-throughput inference. Implement and maintain changes in high-performance inference engines including kernel backends, speculative decoding, quantization, etc. Profile and optimize performance across GPU, networking, and memory layers to improve latency, throughput, and cost. Design and operate RL and post-training pipelines, jointly optimizing algorithms and systems, and making RL and post-training workloads more efficient with inference-aware training loops. Use these pipelines to train, evaluate, and iterate on frontier models on top of the inference stack. Co-design algorithms and infrastructure so that objectives, rollout collection, and evaluation are tightly coupled to efficient inference, identifying bottlenecks across various layers. Run ablations and scale-up experiments to understand trade-offs between model quality, latency, throughput, and cost, feeding insights back into model, RL, and system design. Profile, debug, and optimize inference and post-training services under real production workloads. Drive roadmap items requiring engine modification including changing kernels, memory layouts, scheduling logic, and APIs. Establish metrics, benchmarks, and experimentation frameworks to validate improvements rigorously. Provide technical leadership by setting technical direction for cross-team efforts, mentoring engineers and researchers on full-stack ML systems work and performance engineering.

View detail

The responsibilities for the Global Hardware Sourcing & Supply Manager role include advancing inference efficiency end-to-end by designing and prototyping algorithms, architectures, and scheduling strategies for low-latency, high-throughput inference. The role involves implementing and maintaining changes in high-performance inference engines, profiling and optimizing performance across GPU, networking, and memory layers to improve latency, throughput, and cost. It also requires unifying inference with RL/post-training by designing and operating RL and post-training pipelines and making these workloads more efficient with inference-aware training loops. The role includes training, evaluating, and iterating on frontier models using these pipelines, co-designing algorithms and infrastructure to tightly couple objectives, rollout collection, and evaluation with efficient inference, and quickly identifying bottlenecks across various components. Running ablations and scale-up experiments to understand trade-offs between model quality, latency, throughput, and cost, and owning critical production-scale systems by profiling, debugging, optimizing inference and post-training services are also key responsibilities. The role involves driving roadmap items that require engine modifications, establishing metrics, benchmarks, and experimentation frameworks, and providing technical leadership by setting technical direction for cross-team efforts and mentoring engineers and researchers on full-stack ML systems and performance engineering.

View detail

The Senior Staff Engineer for the Model LifeCycle team at Crusoe is responsible for building a comprehensive managed platform for the entire application development lifecycle with a focus on Machine Learning models including Large Language Models (LLMs). Responsibilities include managing fine-tuning systems for large foundation models such as SFT, PEFT, LoRA, and adapters with multi-node orchestration, checkpointing, failure recovery, and cost-efficient scaling. They implement and maintain end-to-end training pipelines for LLMs, distillation and reinforcement learning pipelines including preference optimization, policy optimization, and reward modeling, as well as manage agent execution infrastructure. They also manage dataset, model, and experiment management tasks including versioning, lineage, evaluation, and reproducible fine-tuning at scale. Additionally, they work closely with product, business, and platform teams to shape core abstractions and APIs, influence architectural decisions around training runtimes, scheduling, storage, and model lifecycle management, contribute to and engage with the open-source LLM ecosystem, and take ownership in designing and building core systems from first principles.

View detail

The Staff Software Engineer for the Model LifeCycle team is responsible for building a comprehensive managed platform for the application development lifecycle with a focus on Machine Learning models, including Large Language Models (LLMs). Responsibilities include contributing to fine-tuning systems for large foundation models, implementing and maintaining end-to-end training pipelines for Large Language Models, contributing to distillation and reinforcement learning pipelines, developing and maintaining agent execution infrastructure, and implementing features for dataset, model, and experiment management such as versioning, lineage, evaluation, and reproducible fine-tuning at scale. The role also involves working closely with Principal Engineers, product, business, and platform teams to implement core abstractions and APIs, contributing to architectural decisions around training runtimes, scheduling, storage, and model lifecycle management, and engaging with the open-source LLM ecosystem. This position offers significant scope for ownership and contribution to the design of core systems.

View detail

Engineer the systems required to train foundation models at scale to maximize hardware utilization and training throughput on large-scale GPU clusters. Profile training loops using PyTorch Profiler, Nsight Systems and Nsight Compute to identify system- and kernel-level bottlenecks and maximize model throughput. Configure and tune composite parallelism strategies such as tensor parallelism (TP), data parallelism (DP), hybrid sharded data parallel (HSDP/FSDP), and expert parallelism (EP) to optimize load balance, minimize critical-path bottlenecks, and manage communication-to-computation trade-offs for large-scale large language model (LLM) training. Collaborate with AI Researchers to define model architectures that enhance hardware efficiency without compromising convergence.

View detail

As a modeling lead for the AI lab, you will be responsible for defining the technical roadmap for the team and supporting the modeling needs across the organization. You will define and establish best practices to manage the model life cycle, from data acquisition to deployment, and build tools and platforms to facilitate building and deploying ML models on different devices with specific constraints. You will work closely with different teams across the organization to support their modeling needs, translating high level user needs to specific modeling requirements, creating plans, and technically driving the team to execute on those. Responsibilities also include defining and driving AI Lab technical strategy in support of HP's AI roadmap, owning decisions across models, runtimes, inference engines, and optimization. Lead the device AI strategy including model compression, quantization, distillation, and hardware aware optimization across CPUs, GPUs, NPUs, and TPUs. Architect and evolve tooling and platforms supporting the full model lifecycle from data and training through evaluation, deployment, and monitoring. Establish standards and evaluation frameworks to ensure high quality, safe, and performant Gen AI models in production. Partner with cross functional leaders and teams to align technical direction with product and hardware strategy. Mentor a small group of senior engineers while operating as a hands-on technical leader who sets direction and moves quickly.