Data Scientist, Core Experimentation
WFA Digital Insight
The demand for skilled data scientists in the AI era has skyrocketed, with a 25% increase in job openings in the past year. As companies like Openai continue to push the boundaries of AI innovation, the need for experts in experimentation methodology, data infrastructure, and causal inference has never been more pressing. With a strong focus on statistical correctness and pragmatic solutions, this role offers a unique opportunity for data scientists to shape the future of experimentation. Candidates should be prepared to showcase their expertise in statistics, causal inference, and online experimentation methodology, as well as their ability to communicate complex concepts to both technical and non-technical audiences.
Job Description
About the Role
The Data Scientist, Core Experimentation role at Openai is a highly technical individual contributor position that will play a critical role in shaping the future of experimentation in the AI era. As a key member of the Statsig team, you will work closely with product, engineering, and infrastructure teams to ensure that experiments are trustworthy, statistically rigorous, and scalable to the needs of frontier AI products. Your day-to-day responsibilities will include driving the statistical direction and technical strategy for Openai's experimentation platform, designing and improving experimentation methodologies, and building pragmatic solutions to real-world experimentation challenges.The Statsig team operates at the intersection of experimentation methodology, data infrastructure, causal inference, and product analytics, and is committed to helping teams make better decisions through reliable experimentation. As a Data Scientist on this team, you will have the opportunity to work on some of the hardest problems in online experimentation, including sample ratio mismatch detection, variance reduction, bias mitigation, metric design, triggered analysis, heterogeneous treatment effects, sequential testing, and experimentation in complex ML systems.
What You Will Do
- Drive the statistical direction and technical strategy for Openai's experimentation platform
- Design and improve experimentation methodologies used across product and research teams
- Build pragmatic solutions to real-world experimentation challenges, balancing rigor with operational simplicity
- Improve the reliability and trustworthiness of experiment results, including detection and prevention of bias, logging issues, and data quality failures
- Develop scalable analytical systems and pipelines in Python and distributed compute environments
- Partner with engineers and product teams to improve experiment design, metric quality, and decision-making practices
- Lead investigations into complex experimentation anomalies and measurement failures
- Establish best practices for experimentation governance, interpretation, and statistical correctness
- Mentor other data scientists and raising the overall technical bar for experimentation and causal inference
- Collaborate with cross-functional teams to develop and implement new experimentation methodologies and tools
- Stay up-to-date with industry trends and emerging technologies in experimentation and data science
What We Are Looking For
- Experience building, scaling, or operating experimentation platforms at a large technology company
- Deep expertise in statistics, causal inference, and online experimentation methodology
- Strong understanding of practical experimentation challenges in production systems
- Experience with areas such as variance reduction, CUPED, sequential testing, SRM detection, metric design, or heterogeneous effects
- Strong coding and systems skills in Python and large-scale data processing frameworks (e.g. Spark)
- Experience designing analytical data models and scalable experimentation pipelines
- Ability to communicate complex statistical concepts clearly to technical and non-technical audiences
- Track record of influencing technical strategy through hands-on technical leadership
- Experience in large-scale product experimentation, ML experimentation, ranking systems, marketplace systems, or similar high-scale experimentation domains
- Strong understanding of data infrastructure and data quality
- Experience with cloud-based data platforms and tools
Nice to Have
- Experience with machine learning algorithms and models
- Knowledge of data visualization tools and techniques
- Experience with Agile development methodologies
- Certification in data science or a related field
- Experience with experimentation platforms such as Statsig or Optimize
Benefits and Perks
- Competitive salary and equity package
- Comprehensive health, dental, and vision insurance
- 401(k) matching program
- Flexible PTO and vacation policy
- Remote work stipend and equipment budget
- Opportunities for professional growth and development
- Access to cutting-edge technologies and tools
- Collaborative and dynamic work environment
- Recognition and reward programs for outstanding performance
How to Stand Out
- Be prepared to showcase your expertise in statistics, causal inference, and online experimentation methodology, and provide specific examples of your experience in these areas.
- Highlight your ability to communicate complex statistical concepts to both technical and non-technical audiences, and provide examples of how you have done so in the past.
- Emphasize your experience with large-scale data processing frameworks, such as Spark, and your ability to design and implement scalable analytical systems.
- Be prepared to discuss your experience with experimentation platforms, such as Statsig or Optimize, and your understanding of data infrastructure and data quality.
- Showcase your ability to influence technical strategy through hands-on technical leadership, and provide examples of how you have done so in the past.
- Be prepared to discuss your experience with machine learning algorithms and models, and your knowledge of data visualization tools and techniques.
- Highlight your certification in data science or a related field, and your experience with Agile development methodologies.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.