Member of Technical Staff (Data Scientist/Engineer, Online Metrics)
Data & Analytics
WFA Digital Insight
The demand for skilled data scientists in the digital marketing sector has grown exponentially, with a 25% increase in job openings over the past year. As companies like Perplexity continue to push the boundaries of AI-driven search engines, professionals with expertise in online metrics and data analysis are in high demand. With its innovative approach to search technology, Perplexity stands out as a company that values data-driven decision making. Before applying, candidates should be prepared to demonstrate their technical skills, particularly in Python and SQL, as well as their ability to communicate complex findings to product teams.
Job Description
About the Role
As a Data Scientist/Engineer on the Answer Quality team at Perplexity, you will play a crucial role in developing and implementing novel online metrics to measure the quality of user interactions. Your work will directly impact the company's ability to provide reliable, high-quality answers to its tens of millions of users. You will be part of a small, high-impact team that collaborates closely with product and search teams to drive product decisions. The Answer Quality team is responsible for ensuring that Perplexity's prompts, tools, and specialized datasets provide the best possible experience for users. As a key member of this team, you will be responsible for analyzing user behavior and developing metrics that accurately predict user satisfaction. Perplexity's innovative approach to search technology, which combines the power of large language models with specialized data sources, requires a deep understanding of both technical and product aspects of search.What You Will Do
- Derive online signals from user interactions to bridge the gap between changes in answer quality and observed user behavior
- Design and implement novel online metrics to be tracked both in A/B testing and on product health dashboards
- Analyze experimental results to validate these metrics and ensure they accurately predict user satisfaction
- Build and maintain the data pipelines that calculate these metrics at scale
- Deliver actionable quality signals to Search, Product, and model training teams
- Communicate findings and bring clarity through close collaboration with Product and Search teams
- Develop and maintain datasets to support the development of new metrics and models
- Work closely with cross-functional teams to integrate metrics into product decisions
- Stay up-to-date with industry trends and developments in online metrics and data analysis
What We Are Looking For
- MS in a technical field or equivalent experience
- 4+ years of experience working as a Data Scientist, Analytics Engineer, or related role
- Experience working on search, recommendation, or LLM-based products
- Strong proficiency in Python and SQL
- Deep knowledge of statistical analysis
- Experience with Business Intelligence (BI) tools for visualization and reporting
- Ability to communicate complex technical concepts to non-technical stakeholders
- Experience working in an agile development environment
- Strong problem-solving skills and attention to detail
Nice to Have
- Proficiency with Apache Spark and Databricks
- Experience with the development or validation of LLM-as-a-judge systems
- Prior work supporting customer-facing products at scale
- Experience with cloud-based data platforms
- Familiarity with DevOps practices and tools
Benefits and Perks
- Competitive salary and benefits package
- Opportunity to work on a high-impact project with a talented team
- Flexible working hours and remote work options
- Professional development opportunities, including training and conference sponsorships
- Access to cutting-edge technologies and tools
- Collaborative and dynamic work environment
- Recognition and rewards for outstanding performance
- Comprehensive health insurance and retirement savings plan
- Generous paid time off and holiday policy
How to Stand Out
- Develop a strong portfolio that showcases your experience with online metrics and data analysis, highlighting specific projects and accomplishments.
- Practice communicating complex technical concepts to non-technical stakeholders, as this is a key requirement for the role.
- Be prepared to discuss your experience with A/B testing and experiment design, as well as your ability to analyze and interpret results.
- Show a willingness to learn and adapt to new technologies and tools, such as Apache Spark and Databricks.
- Highlight your ability to work collaboratively with cross-functional teams, including product and search teams.
- Research Perplexity's company culture and values, and be prepared to discuss how your own values and approach to work align with theirs.
- Prepare to back up your claims with specific examples and metrics, demonstrating the impact of your work on previous projects.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.