AI Software Engineer Jobs

View detail

As the first US based Engineer, the role entails acting as the technical bridge between the product vision and customer reality. Responsibilities include designing, architecting, and shipping full-stack features that solve customer compliance challenges, owning the technical relationship with key customers by implementing solutions, gathering requirements, and translating feedback into product improvements. The engineer will build scalable services and APIs for the LLM compliance platform, make high-impact technical decisions quickly while being accountable to engineering standards and customers, challenge assumptions about what and how to build, and shape the product roadmap and engineering practices as the company scales from Series A to market leadership. The work combines coding, customer engagement, and steering product direction with high autonomy and collaboration with the founding team.

View detail

The Software Engineer on the Workspace & Observability Team at Anyscale is responsible for building user-facing application features for the Anyscale AI platform, focusing on the backend to implement the core business logic of these features. Responsibilities include interacting with users to understand their requirements, designing and implementing features, maintaining and improving these features over time, and working on observability tools that help users monitor and debug AI applications running on distributed clusters. Specific projects may involve developing the Ray Dashboard observability tool, library-specific observability tools like the Ray Train and Ray Serve dashboards, a unified log viewer for querying logs across a Ray cluster, and anomaly detection features to automatically identify and suggest fixes for performance bottlenecks or bugs. The role also involves collaborating with distributed systems and machine learning experts, communicating work through talks, tutorials, and blog posts, and contributing to building and shaping the company.

View detail

Design, architect, and operate scalable services and APIs that power the LLM compliance platform. Architect how AI insights are surfaced to users, ensuring the system is robust, fast, and intuitive. Make high-impact technical decisions quickly. Challenge "why" and "how" to ensure delivery of the best possible experience for users. Shape engineering culture, standards, and tooling as the company grows. Own end-to-end technical decisions including designing systems, architecting solutions, shipping to production, and iterating based on customer feedback.

View detail

Work on a small, high-caliber team building AI products for clients, from requirements gathering and prototyping through system design, development, testing, and deployment. Own features end-to-end and develop domain expertise across a range of AI use cases. Spend most of the time coding and frequently interact with clients to ensure the solutions meet their needs.

View detail

You will work on a small, high-caliber team building AI products for clients, setting technical direction, writing code, and serving as the go-to person when challenges arise. Spend approximately 75% of your time coding and 25% interacting with clients, including CTOs, to understand problems, evaluate tradeoffs, and ensure solutions meet their needs.

View detail

As a member of the Bots team, design, build, and scale systems that enhance user engagement with the AI-powered platform, including bot chat orchestration, AI image generation, AI video generation, and tooling for managing these features. Collaborate with cross-functional teams like product managers, designers, and data specialists to deliver high-quality, performant, and maintainable features. Experiment with and integrate new AI image, video, and voice generation technologies. Build tooling and infrastructure around various AI technologies. Gain exposure to the architecture and operations of a fast-growing social AI product. Contribute expertise to evolve team processes and technical infrastructure, ensuring scalability and reliability.

View detail

Work on projects such as developing a product that root causes KTLO work and recommends solutions, building a software catalog that works for monoliths and is user-friendly, and helping protect engineering focus time by systemically solving sources of distraction or mental load with AI.

View detail

As a Staff Software Engineer on the Core Infrastructure team at Harvey, your responsibilities include designing and building scalable, fault-tolerant infrastructure systems that power Harvey's AI platform across multiple cloud regions. You will own and evolve the multi-cloud infrastructure (Azure, GCP), including Kubernetes orchestration, networking, and container management. You will lead technical initiatives focused on observability, incident response, and operational excellence, building systems for rapid detection and resolution of issues. Architecting and optimizing distributed systems for reliability, including load balancing, quota management, and failover mechanisms, will be part of your role. You will partner with Product Engineering and Security teams to ensure infrastructure accelerates product development, drive infrastructure-as-code practices using tools like Terraform and Pulumi for reproducible deployments, and mentor engineers through code reviews, design reviews, and technical leadership. Representative projects include designing model proxy architecture for handling inference requests, building distributed rate limiting and quota management systems, architecting multi-region deployment strategies for data residency compliance, developing observability infrastructure with SLA monitoring and cost tracking, and leading CI/CD pipeline evolution to improve velocity and stability.

View detail

Develop systems and tooling to measure, monitor, and improve token throughput across first-party and partner-owned compute environments. Support performance benchmarking, tokenomics analysis, and model porting across heterogeneous infrastructure environments. Build tooling to integrate external or partner infrastructure into OpenAI’s internal compute, observability, and workload management systems. Develop and monitor operational metrics including billing, usage, SLAs, utilization, reliability, and throughput. Identify bottlenecks across hardware, networking, software, and workload enablement that prevent capacity from becoming productive tokens. Partner with compute, infrastructure, networking, finance, and operations teams to translate raw capacity into usable workload-serving capacity. Build dashboards, automation, and reporting systems that provide clear visibility into TaaS capacity, performance, and business outcomes.

View detail

As a Software Engineer on the Coding Pod, you will build the data infrastructure and pipelines that power frontier AI coding models. Responsibilities include designing and building scalable data pipelines for generating, transforming, and validating large-scale coding datasets; developing systems for task generation, dataset curation, and quality assurance, including automated and human-in-the-loop evaluation workflows; integrating with developer ecosystems such as GitHub and building tooling to support real-world coding environments; working with containerized environments like Docker to safely execute and evaluate code at scale; building backend systems and APIs that power dataset delivery and model evaluation pipelines; collaborating closely with ML researchers, product managers, and other engineers to define evaluation methodologies and improve dataset quality; implementing automated grading, benchmarking, and assessment systems for coding tasks; debugging and optimizing pipeline performance, reliability, and scalability across distributed systems; and contributing to architectural decisions around data infrastructure, evaluation systems, and pipeline orchestration.