System Software Engineer, Diagnostics & Test

About the Team

The scaling team collaborates to design AI supercomputing systems, pushing the boundaries of infrastructure innovation.

About the Role

We're seeking a founding Systems Software Engineer to join our scaling team. In this role, you will define and develop diagnostics tooling as well as the associated infrastructure for automating multi-node diagnostics at scale. You will work on diagnostics for various components of our compute infrastructure, including board management controllers, compute nodes, and networking elements.

This role involves close collaboration with hardware teams, software teams, and systems manufacturers to ensure only healthy machines enter the fleet, and to empower other engineers to diagnose and debug problems using your tooling. You will also review and work closely with third-party vendors to help guide their development and delivery of artifacts needed to build our internal diagnostics tooling.

Responsibilities

• Design, develop, and maintain diagnostics software infrastructure for automating multi-node system testing at scale.

• Build and integrate diagnostic test tools for both at-scale automation and manual lab debugging.

• Collaborate with hardware teams and external vendors to define and build diagnostics and qualification software for compute infrastructure.

• Help with bringups and lab debugging using your tooling.

• Lead and define diagnostics software requirements with vendors, review deliverables, and manage third-party software integration.

• Monitor vendor software development, ensuring adherence to technical standards, timelines, and quality metrics.

• Collaborate cross-functionally with hardware, system architecture, power, and cooling teams.

• Support integration and readiness of compute systems through scalable software solutions.

• Enable the utmost reliability and health of our serving machines by ensuring systems entering and operating in our fleet are healthy thanks to rigorous diagnostics testing.

Qualifications

• Experience with building fleet-level system test automation and diagnostic systems.

• Strong Python, C, C++, and Linux shell scripting skills. Rust experience is also a big plus.

• Knowledge of hardware subsystems in compute environments and the common failure-modes that need to be tested or debugged.

• Experience working with hardware engineers to understand their test and diagnostics requirements and building associated tooling.

• Proven track record defining and managing vendor relationships and deliverables.

• Understanding of system monitoring in high-performance computing environments and fleet management software.

• Some experience with embedded systems, including bootloaders, drivers, and hardware-software interfacing.

• Ability to thrive in ambiguity and implement things from scratch.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

OpenAI Affirmative Action and Equal Employment Opportunity Policy Statement

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Compensation Range: $310K - $465K