Tokens-as-a-Service (Taas) Software Engineer

About the Role

As a TaaS Engineer, you will be responsible for developing systems and tooling to measure, monitor, and improve token throughput across first-party and partner-owned compute environments. This includes building tooling to integrate external or partner infrastructure into OpenAI's internal compute, observability, and workload management systems. You will also work on developing and monitoring operational metrics, including billing, usage, SLAs, utilization, reliability, and throughput.

The TaaS Engineer will be part of a team that is pushing the boundaries of what is possible with AI. OpenAI is committed to ensuring that general-purpose artificial intelligence benefits all of humanity, and the TaaS Engineer will play a critical role in achieving this mission.

What You Will Do

• Develop systems and tooling to measure, monitor, and improve token throughput across first-party and partner-owned compute environments.
• Support performance benchmarking, tokenomics analysis, and model porting across heterogeneous infrastructure environments.
• Build tooling to integrate external or partner infrastructure into OpenAI's internal compute, observability, and workload management systems.
• Develop and monitor operational metrics, including billing, usage, SLAs, utilization, reliability, and throughput.
• Identify bottlenecks across hardware, networking, software, and workload enablement that prevent capacity from becoming productive tokens.
• Partner with compute, infrastructure, networking, finance, and operations teams to translate raw capacity into usable workload-serving capacity.
• Build dashboards, automation, and reporting systems that provide clear visibility into TaaS capacity, performance, and business outcomes.
• Collaborate with cross-functional teams to design and implement scalable and efficient infrastructure solutions.
• Troubleshoot issues with token throughput, infrastructure, and workload performance.
• Develop and maintain documentation for the TaaS system, including architecture diagrams and technical notes.

What We Are Looking For

• Strong software engineering background with experience building systems, tooling, automation, or infrastructure platforms.
• Experience working across compute infrastructure, distributed systems, performance engineering, or production operations.
• Ability to reason about token throughput, utilization, benchmarking, infrastructure efficiency, and workload performance.
• Comfortable integrating external systems or partner environments into internal infrastructure stacks.
• Strong analytical and debugging skills across hardware, networking, software, and operational domains.
• Experience with programming languages such as Python, Java, or C++.
• Familiarity with cloud-based infrastructure, including AWS, GCP, or Azure.
• Knowledge of containerization using Docker, Kubernetes, or other container orchestration tools.
• Understanding of agile development methodologies and version control systems such as Git.

Nice to Have

• Experience with GPU clusters, AI infrastructure, performance benchmarking, or workload optimization.
• Familiarity with model porting, inference/training workloads, token economics, or compute efficiency analysis.
• Experience building monitoring systems for billing, usage, SLAs, utilization, or infrastructure reliability.
• Background in systems engineering, infrastructure software, observability, distributed systems, or platform engineering.
• Knowledge of security best practices and compliance frameworks.

Benefits and Perks

• Competitive salary and equity package.
• Comprehensive health, dental, and vision insurance.
• Flexible PTO policy and paid holidays.
• Remote work stipend and home office setup support.
• Professional development opportunities, including conferences, training, and mentorship.
• Access to cutting-edge technology and tools.
• Collaborative and dynamic work environment with a team of experienced professionals.
• Opportunity to work on high-impact projects that are changing the face of AI research and development.

• Tip: Showcase your experience with cloud-based infrastructure and containerization using tools like Docker and Kubernetes.
• To stand out, highlight your analytical and debugging skills, as well as your ability to reason about token throughput and workload performance.
• Be prepared to discuss your experience with agile development methodologies and version control systems like Git.
• Tip: Familiarize yourself with OpenAI's mission and values, and be prepared to discuss how your skills and experience align with the company's goals.
• When preparing your portfolio, include examples of your work with systems engineering, infrastructure software, or distributed systems.
• Tip: Research the current market rate for your salary and be prepared to negotiate based on your experience and qualifications.