ML Infrastructure Engineer Jobs

Discover the latest remote and onsite ML Infrastructure Engineer roles across top active AI companies. Updated hourly.

Check out 20 new ML Infrastructure Engineer opportunities posted on The Homebase

Member of Technical Staff - ML Training Systems

New
Top rated
Modal
Full-time
Full-time
Posted

The role involves contributing to open-source projects and evolving Modal's infrastructure to train the next generation of language models. The focus is on strong engineering skills with experience in training production machine learning models.

$150,000 – $350,000
Undisclosed
YEAR

(USD)

New York, United States
Maybe global
Hybrid

Staff Software Engineer, ML Infrastructure

New
Top rated
Decagon
Full-time
Full-time
Posted

Design and build distributed training platforms for LLM and multimodal fine-tuning and post-training at scale; implement and integrate state-of-the-art training algorithms into production pipelines; own inference architecture and multi-provider routing, including failover and optimization; research and implement inference optimizations including quantization, speculative decoding, and batching strategies; lead initiatives to improve latency and cost efficiency across the training and serving stack; build evaluation and experimentation infrastructure that enables rapid, reliable iteration; drive technical direction, mentor engineers, and establish best practices for ML infrastructure.

$300,000 – $430,000
Undisclosed
YEAR

(USD)

San Francisco, United States
Maybe global
Onsite

Staff Strategic Sourcing Manager (Hardware)

New
Top rated
Together AI
Full-time
Full-time
Posted

Advance inference efficiency end-to-end by designing and prototyping algorithms, architectures, and scheduling strategies for low-latency, high-throughput inference. Implement and maintain changes in high-performance inference engines including kernel backends, speculative decoding, and quantization. Profile and optimize performance across GPU, networking, and memory layers to improve latency, throughput, and cost. Design and operate RL and post-training pipelines such as RLHF, RLAIF, GRPO, DPO-style methods, and reward modeling, optimizing algorithms and systems jointly. Make RL and post-training workloads more efficient with inference-aware training loops including async RL rollouts and speculative decoding techniques. Use these pipelines to train, evaluate, and iterate on frontier models on top of the inference stack. Co-design algorithms and infrastructure to tightly couple objectives, rollout collection, and evaluation with efficient inference, quickly identifying bottlenecks across the training engine, inference engine, data pipeline, and user-facing layers. Run ablations and scale-up experiments to understand trade-offs between model quality, latency, throughput, and cost, feeding insights back into model, RL, and system design. Profile, debug, and optimize inference and post-training services under real production workloads. Drive roadmap items that require engine modification such as changing kernels, memory layouts, scheduling logic, and APIs. Establish metrics, benchmarks, and experimentation frameworks to validate improvements rigorously. Provide technical leadership including setting technical direction for cross-team efforts at the intersection of inference, RL, and post-training, and mentoring engineers and researchers on full-stack ML systems work and performance engineering.

$200,000 – $280,000
Undisclosed
YEAR

(USD)

San Francisco
Maybe global
Onsite

Principal Engineer, AI Model LifeCycle

New
Top rated
Crusoe
Full-time
Full-time
Posted

The Principal Software Engineer for the Model LifeCycle team is responsible for managing fine-tuning systems for large foundation models including multi-node orchestration, checkpointing, failure recovery, and cost-efficient scaling. They implement and maintain end-to-end training pipelines for Large Language Models and distillation and reinforcement learning pipelines such as preference optimization, policy optimization, and reward modeling. They work on agent execution infrastructure and oversee dataset, model, and experiment management including versioning, lineage, evaluation, and reproducible fine-tuning at scale. The role involves close collaboration with product, business, and platform teams to shape core abstractions and APIs, influence long-term architectural decisions related to training runtimes, scheduling, storage, and model lifecycle management, and contribute to the open-source LLM ecosystem. The position includes significant ownership in designing and building core systems from first principles.

$260,000 – $326,000
Undisclosed
YEAR

(USD)

San Francisco, United States
Maybe global
Onsite

Staff Machine Learning Engineer

New
Top rated
Adaptive Security
Full-time
Full-time
Posted

Define Adaptive's ML strategy including where ML should be applied across products, required infrastructure, and build vs. buy decisions. Design and build production ML systems end-to-end including data pipelines, model training, evaluation frameworks, and inference serving. Establish evaluation methodology to measure model quality, catch regressions, and make data-driven decisions about model changes. Own the strategy for acquiring and formatting necessary data, including labeling, feedback loops, and model improvement over time. Partner with product engineers to integrate ML into the product by writing production code and working within existing codebase. Help build and lead the ML team as scope grows.

Undisclosed

()

New York, United States
Maybe global
Onsite

Machine Learning Engineer, Distributed Data Systems

New
Top rated
OpenAI
Full-time
Full-time
Posted

Design, build, and maintain data infrastructure systems such as distributed compute, data orchestration, distributed storage, streaming infrastructure, and machine learning infrastructure while ensuring scalability, reliability, and security. Ensure the data platform can scale by orders of magnitude while remaining reliable and efficient. Partner with researchers to deeply understand requirements and translate them into production-ready systems. Harden, optimize, and maintain critical data infrastructure systems that power multimodal training and evaluation.

$295,000 – $445,000
Undisclosed
YEAR

(USD)

San Francisco, United States
Maybe global
Hybrid

Principal Machine Learning Engineer

New
Top rated
PhysicsX
Full-time
Full-time
Posted

The role involves building a platform used by Data Scientists and Simulation Engineers to build, train, and deploy Deep Physics Models. The candidate will work on a focused, stream-aligned, and cross-functional team that includes back-end, front-end, and design members, empowered to make its own implementation decisions towards meeting its objectives. Responsibilities include gathering and leveraging domain knowledge and experience from the Data Scientists and Simulation Engineers using the product, taking ownership of work from implementation to production, ensuring quality, scalability, and observability at every step, which includes testing, containerization, continuous integration and delivery, authentication, authorization, telemetry, observability, and monitoring.

Undisclosed

()

Singapore
Maybe global
Hybrid

Machine Learning Engineer: ML Infra and Model Optimization

New
Top rated
Genies
Intern
Full-time
Posted

Develop and deploy LLM agent systems within the AI-powered avatar framework. Design and implement scalable and efficient backend systems to support AI applications. Collaborate with AI and NLP experts to integrate LLM and LLM-based systems and algorithms into the avatar ecosystem. Work with Docker, Kubernetes, and AWS for AI model deployment and scalability. Contribute to code reviews, debugging, and testing to ensure high-quality deliverables. Document work for future reference and improvement.

$40 – $50 / hour
Undisclosed
HOUR

(USD)

Los Angeles, United States
Maybe global
Hybrid

NPI Engineer

New
Top rated
Figure AI
Full-time
Full-time
Posted

Design, deploy, and maintain Figure's training clusters. Architect and maintain scalable deep learning frameworks for training on massive robot datasets. Work together with AI researchers to implement training of new model architectures at a large scale. Implement distributed training and parallelization strategies to reduce model development cycles. Implement tooling for data processing, model experimentation, and continuous integration.

$150,000 – $350,000
Undisclosed
YEAR

(USD)

San Jose, United States
Maybe global
Onsite

Helix Data Creator

New
Top rated
Figure AI
Full-time
Full-time
Posted

Design, deploy, and maintain Figure's training clusters. Architect and maintain scalable deep learning frameworks for training on massive robot datasets. Work together with AI researchers to implement training of new model architectures at a large scale. Implement distributed training and parallelization strategies to reduce model development cycles. Implement tooling for data processing, model experimentation, and continuous integration.

$150,000 – $350,000
Undisclosed
YEAR

(USD)

Spartanburg, United States
Maybe global
Onsite

Want to see more ML Infrastructure Engineer jobs?

View all jobs

Access all 4,256 remote & onsite AI jobs.

Join our private AI community to unlock full job access, and connect with founders, hiring managers, and top AI professionals.
(Yes, it’s still free—your best contributions are the price of admission.)

Frequently Asked Questions

Have questions about roles, locations, or requirements for ML Infrastructure Engineer jobs?

Question text goes here

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

[{"question":"What does a ML Infrastructure Engineer do?","answer":"ML Infrastructure Engineers design, build, and maintain systems that support machine learning workflows from development to production. They create scalable platforms for model training and serving, implement distributed training systems, and develop monitoring solutions to track model performance. These engineers also build data pipelines, optimize ML systems for performance, and implement automated testing and deployment processes while collaborating with data scientists and researchers to productionize ML models."},{"question":"What skills are required for ML Infrastructure Engineer?","answer":"ML Infrastructure Engineers need strong programming skills in Python and sometimes Go, Rust, or C++. Proficiency with ML frameworks like PyTorch and TensorFlow is essential, alongside expertise in cloud platforms (AWS, GCP), containers (Docker), and orchestration (Kubernetes). They should understand distributed systems, data engineering concepts, and model serving techniques. Experience with infrastructure-as-code tools and monitoring systems rounds out the technical requirements, complemented by problem-solving abilities and collaboration skills."},{"question":"What qualifications are needed for ML Infrastructure Engineer role?","answer":"Most ML Infrastructure Engineer positions require a Bachelor's or Master's degree in Computer Science or related field, plus 4-5+ years of experience building production ML systems. Employers typically expect demonstrable experience with cloud platforms, containerization tools, and ML frameworks. Strong understanding of system-level software, machine learning concepts, and resource utilization is necessary. Experience with distributed systems and high-throughput workloads is highly valued, especially for senior positions."},{"question":"What is the salary range for ML Infrastructure Engineer job?","answer":"The research provided doesn't specify salary ranges for ML Infrastructure Engineer jobs. Compensation typically varies based on factors like location, company size, experience level, and specific technical expertise. Organizations like Anthropic, Scale AI, Apple, and other technology companies actively hiring for these positions likely offer competitive compensation packages reflecting the specialized nature of ML infrastructure skills and the current market demand."},{"question":"How long does it take to get hired as a ML Infrastructure Engineer?","answer":"The hiring timeline for ML Infrastructure Engineer positions isn't specified in the provided research. The process typically includes technical interviews focused on systems design, ML fundamentals, and programming skills. Given the specialized nature of the role, companies often conduct thorough evaluations of candidates' experience with production ML systems, distributed computing, and relevant technologies. The specialized requirements may extend the hiring process compared to more general engineering roles."},{"question":"Are ML Infrastructure Engineer job in demand?","answer":"Yes, ML Infrastructure Engineer jobs show strong demand based on active openings at major companies like DataXight, Scale AI, Anthropic, Apple, and Character.AI. The field is growing particularly in specialized areas such as LLM serving infrastructure, on-device ML optimization, and safety-critical ML systems. These positions are distributed across major tech hubs with opportunities ranging from mid-level to senior roles, reflecting industry's increasing need for engineers who can build reliable ML systems at scale."}]