Research Engineer - Evals
Build the eval harness for AGI covering model capability, agentic behavior, on-device performance, and end-user experience. Own eval suites gating every model and agent release, including capability, behavior, regressions, and human-rated rubrics. Maintain dashboards and tooling to facilitate fast researcher experiment loops and informed leadership decisions. Set and uphold the criteria for what counts as ready to ship. Assist research by ensuring measurements align with goals. Aid product engineers by instrumenting real-user behavior on devices. Support partnerships by translating performance improvements into measurable terms for OEM partners.
Senior Scientist, Analytical Chemistry
The Senior Scientist is responsible for owning the end-to-end analytical strategy for GC-MS-based programs, including method design, validation frameworks, and data quality standards for targeted and untargeted analyses. They define and evolve sample preparation methodologies for headspace, liquid-phase, and solid-phase extraction of fragrance compounds from complex matrices and consumer products. They maintain and improve Osmo's high-throughput analytical pipeline, ensuring data integrity, reproducibility, and compatibility with downstream machine learning workflows. The role involves partnering with the Platform and ML teams as the chemistry-side technical owner of the data interface, determining methods and procedures for new analytical assignments independently while coordinating execution across team members and collaborating functions. They enforce high standards of scientific rigor and data quality, mentor and develop junior and mid-level scientists, establish best practices, review work for scientific integrity, and elevate the team’s overall analytical capability. Additional responsibilities include writing, editing, and auditing analytical and experimental protocols, serving as an internal expert resource and external-facing collaborator for analytical chemistry questions across Osmo’s scientific and commercial programs.
Researcher, Context - Agent Post-Training
As a Context Researcher on the Agent Post-Training team, the role involves designing and running experiments to improve the scaling of compute on context. The researcher will own end-to-end improvements to the post-training stack, including reinforcement learning, data pipelines, graders, reward signals, evaluations, diagnostics, and model-behavior analysis. Responsibilities include building evaluations and environments to identify model failures and turning those failures into training data, product fixes, or new research directions. The researcher will partner with Codex and ChatGPT product teams to translate product signals into model improvements and work on early-training and alignment interventions such as data mixtures, objectives, synthetic data, and evaluation loops to shape downstream agent behavior. The role involves deciding which integrations, capabilities, and fixes are ready for major model runs, improving machinery for large-scale training and launch including experiment velocity, reliability, observability, reproducibility, cost, latency, and production readiness. The researcher will take on cross-functional projects involving model training, product infrastructure, and the production agent harness and debug failures in shipped or near-shipped models by developing hypotheses, experiments, and fixes from qualitative behaviors.
Researcher, Connectors - Agent Post-Training
As a member of Agent Post-Training, Connectors, you will teach models how to interface with professional software using code, helping train agents to use code, APIs, tools, and structured integrations to operate across applications like Slack, Google Workspace, GitHub, Notion, Linear, Salesforce, and other core systems. You will design and run experiments to improve agentic model behavior for complex software and plugins, own end-to-end improvements to the post-training stack including RL, data pipelines, graders, reward signals, evaluations, diagnostics, and model behavior analysis, and build evaluations and environments that expose model failures to turn those failures into training data, product fixes, or new research directions. You will partner with product teams to understand user needs and translate product signals into model improvements, work on early-training and alignment interventions such as data mixtures, objectives, synthetic data, and evaluation loops, and decide which integrations and capabilities to include in major model runs. Additionally, you will improve large-scale training and launch infrastructure for experiment velocity, reliability, observability, reproducibility, cost, latency, and production readiness, take on cross-functional projects touching model training, product infrastructure, and the production agent harness, and debug failures in shipped or near-shipped models to develop concrete hypotheses, experiments, and fixes.
Researcher, Computer Use - Agent Post-Training
As a member of Agent Post-Training, Computer Use, you will teach models to operate computers, helping to train models that can navigate browsers and desktops, use tools and applications, reason through complex workflows, collaborate with users and other agents, and complete long-horizon tasks with reliability and judgment. Responsibilities include designing and running experiments to improve agentic model behavior for complex computer use, owning end-to-end improvements to the post-training stack such as reinforcement learning, data pipelines, graders, reward signals, evaluations, diagnostics, and model-behavior analysis. You will build evaluations and environments to identify model failures and convert those into training data, product fixes, or research directions. The role involves partnering with product teams to understand user needs and translate product signals into model improvements, working on early-training and alignment interventions, deciding on suitable integrations and fixes for major model runs, and improving large-scale training and launch machinery regarding experiment velocity, reliability, observability, reproducibility, cost, latency, and production readiness. You will also handle cross-functional projects involving model training, product infrastructure, and production agent harness, debug failures in shipped or near-shipped models, and transform qualitative model behavior into concrete hypotheses, experiments, and fixes.
Researcher, Artifacts - Agent Post-Training
As a member of Agent Post-Training, Artifacts, the role involves training frontier models to produce polished, useful work products such as documents, spreadsheets, slide decks, dashboards, reports, analyses, and other interactive or editable artifacts. Responsibilities include designing and running experiments to improve agentic model behavior for complex software and plugins, owning end-to-end improvements to the post-training stack including reinforcement learning, data pipelines, graders, reward signals, evaluations, diagnostics, and model-behavior analysis. The role involves building evaluations and environments to identify new model failures and converting these failures into training data, product fixes, or new research paths. Collaboration with Codex and ChatGPT product teams to translate product signals into model improvements is required. Other duties include working on early-training and alignment interventions, deciding integration and capability readiness for major model runs, improving machinery for large-scale training and launch regarding experiment velocity, reliability, observability, reproducibility, cost, latency, and production readiness, and undertaking cross-functional projects that involve model training, product infrastructure, and production agent systems. Debugging hard failures in shipped or near-shipped models and transforming qualitative behaviors into hypotheses, experiments, and fixes is also part of the role.
Applied AI Researcher, Multi-Agent Systems
The Multi-Agent Systems team focuses on designing architectures in which multiple agents coordinate to solve problems that require structured interaction across multiple reasoning processes. Researchers build systems that structure communication, route information, and coordinate decision-making across agents operating with different views of the problem. Researchers investigate the interaction patterns that govern how agents collaborate, studying how agents exchange information, critique and refine each other’s reasoning, and coordinate execution across complex workflows. Their work identifies the mechanics behind effective communication, delegation, and coordination, establishing the design language for how systems of agents can operate as cohesive, high-performing teams, with capabilities that arise from interaction rather than individual performance.
Research Scientist, Safety Post Training
The role involves owning the production outcome and taking full accountability for the long-term performance and reliability of AI use cases deployed across international government agencies. It includes ensuring full-stack integrity by overseeing the end-to-end health of the platform and seamless integration between the AI core and all full-stack components, from APIs to UI, to maintain a responsive and production-ready environment. Responsibilities also cover scaling the feedback loop by building automated systems to monitor model performance and data drift across geographically dispersed environments for appropriate reliability. Managing the technical lifecycle within diverse regulatory frameworks and leading the response for production issues in mission-critical environments to ensure rapid resolution and prevent recurrence are also required. Additionally, the role entails translating deep technical performance metrics into clear insights for senior international government officials and partnering with Engineering and ML teams to influence the technical architecture and decisions of future use cases based on lessons learned in the field.
Research Scientist (Singapore)
Drive foundational research on video generation models, taking ownership across the full research cycle and driving post-training research. Collaborate closely with data, infrastructure, and adjacent modeling teams to translate research findings into durable model improvements. Build and maintain scalable systems for ingesting, preprocessing, and delivering large-scale video data for model training. Design and scale distributed data pipelines for preprocessing, dataset generation, and repeated dataset refreshes. Own workflow orchestration, job scheduling, monitoring, and failure recovery for large-scale data processing jobs. Implement and maintain containerized pipeline infrastructure using Kubernetes or equivalent orchestration systems. Optimize cloud-based data storage and movement across providers (AWS, GCS, or Azure) for cost, throughput, and operational efficiency. Define and implement best practices for dataset storage layout, versioning, caching, retention, and access patterns. Build tooling to support deduplication workflows at scale, including near-dedup pipelines over large video corpora. Research and develop distillation methods for large-scale diffusion and flow-based video generation models, including guidance distillation and adversarial distillation, focusing on preserving or improving generation quality while reducing inference cost. Develop reward models and preference-based fine-tuning pipelines that align video generation quality with human judgments across aesthetics, motion quality, and prompt adherence. Analyze the relationship between base model behavior and post-training outcomes, working with foundation model team to inform pretraining decisions accordingly.
Researcher, Alignment Oversight
As a researcher on the Alignment Oversight team, you will design and run experiments to improve oversight of increasingly capable AI models, involving model training, evaluation design, and research infrastructure. Responsibilities include deploying practical systems for action monitoring, red-teaming, and human-in-the-loop control; developing evaluations for alignment failure modes of frontier models, such as overeagerness and instruction following failures; analyzing deployment data to understand model failures and oversight gaps; developing techniques to feed oversight signals back into training while preserving oversight reliability; producing publishable research advancing alignment science; collaborating with research, product, security, safety, and engineering teams to implement alignment ideas; and rapidly moving from research intuition to working experiments, prototypes, and evidence that inform future model improvements.
Access all 4,256 remote & onsite AI jobs.
Frequently Asked Questions
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.