Associate Director, AI Performance & Operations
WFA Digital Insight
The demand for AI and digital skills in healthcare has grown exponentially, with a 25% increase in remote job postings in the last year alone. As a leader in AI performance and operations, you'll be at the forefront of this shift. With Devoted Health, you'll have the opportunity to shape the future of member interactions and quality monitoring. To succeed in this role, you'll need a unique blend of technical expertise, leadership skills, and intuition for quality in human interactions. Before applying, consider how your experience in high-ownership operator roles, people leadership, and AI tools can help drive meaningful improvements in member experience and outcomes.
Job Description
About the Role
The Associate Director, AI Performance & Operations role is a newly created position at Devoted Health, designed to build and manage AI systems that monitor the quality of every member interaction. You will be responsible for defining what high performance looks like across frontline teams and AI agents, and building the systems that ensure the company hits its goals. This role is critical to the organization's mission, as every interaction with members is an opportunity to earn or lose their trust.As a key member of the leadership team, you will work closely with frontline team leaders to understand what quality looks like in practice and ensure that every tool you build is adopted and helps them meaningfully improve performance. Your expertise in AI and digital skills will be essential in driving the development of AI-native quality and performance operations.
The role will involve working with large datasets, evaluating member interactions in real-time, and building coaching workflows that surface actionable feedback to leaders at all levels. You will also be responsible for visualizing interaction data, evaluation criteria, coaching signals, and performance trends to ensure that the right insights reach the right person at the right time.
What You Will Do
- Embed with frontline teams and their leaders to understand what quality looks like in practice
- Own the performance of high-volume frontline AI agents, across voice and chat
- Build AI-powered quality monitoring systems that drive business outcomes
- Evaluate member interactions in real-time, moving beyond pass/fail scoring to nuanced assessment
- Build coaching workflows that surface actionable feedback to leaders at all levels
- Visualize interaction data, evaluation criteria, coaching signals, and performance trends
- Scale successful systems into reusable frameworks that extend to new teams, interaction types, and channels
- Partner with frontline team leaders to ensure adoption and drive value
- Oversee a small team of operators and agent builders that will drive performance across all frontline agents and teams
- Develop and maintain metrics to measure the effectiveness of AI systems and coaching workflows
- Collaborate with cross-functional teams to ensure alignment and drive business outcomes
What We Are Looking For
- 7+ years of experience in high-ownership operator roles, such as startup strategy, business operations, consulting, or internal tooling PM
- People leadership experience, with a track record of building trust, giving feedback, and creating conditions for teams to do their best work
- Fluency with modern AI tools and platforms, with an active practice of pushing their limits
- Adaptive and curious, with a willingness to learn, teach, lead, and follow
- Systems thinking, with the ability to design frameworks that prevent entire categories of issues
- Instinct for quality in human interactions, with the ability to translate intuition into structured criteria for AI systems
- Strong communication and collaboration skills, with the ability to work with cross-functional teams
- Experience with data analysis, visualization, and reporting
Nice to Have
- Experience with cloud-based AI platforms and tools
- Knowledge of healthcare industry trends and regulations
- Familiarity with agile development methodologies
- Certification in AI, machine learning, or data science
Benefits and Perks
- Competitive salary and benefits package
- Opportunity to work with a leading healthcare company
- Remote work arrangement with flexible hours
- Professional development and growth opportunities
- Access to cutting-edge AI tools and technologies
- Collaborative and dynamic work environment
- Comprehensive health insurance and wellness programs
- Generous paid time off and holidays
How to Stand Out
- Tip: Develop a strong understanding of AI tools and platforms, and be prepared to provide examples of how you've used them to drive business outcomes.
- Ensure your resume and cover letter highlight your experience in people leadership, systems thinking, and data analysis.
- Be prepared to discuss your approach to quality in human interactions and how you would translate intuition into structured criteria for AI systems.
- Showcasing your ability to communicate complex technical concepts to non-technical stakeholders is crucial.
- Prepare to discuss your experience with data visualization and reporting, and how you've used metrics to measure the effectiveness of AI systems.
- Be ready to provide examples of how you've worked with cross-functional teams to drive business outcomes, and how you've handled conflicts or challenges in the past.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.