Staff Machine Learning Engineer

## About the Role As a Staff Machine Learning Engineer at Fullscript, you will be instrumental in developing and deploying innovative AI capabilities that enhance the quality of care provided by clinicians. Your primary focus will be on building and implementing machine learning applications, specifically LLM-powered features, that support clinicians in their decision-making processes. You will work closely with a high degree of autonomy, partnering with various stakeholders including engineering, product, analytics, and medical teams to deliver scalable, reliable, and clinically useful AI experiences.

The role is part of Fullscript's AI team, which is dedicated to shaping the next generation of AI-powered experiences. You will lead the design, development, and deployment of production-ready, multi-turn LLM-powered features, including summarization tools and clinician-facing conversational agents. Your expertise will help define technical direction for prompting, grounding, safety, and orchestration strategies used across clinical AI workflows.

Fullscript is committed to making care smarter and more human, and as a Staff Machine Learning Engineer, you will play a critical role in achieving this mission. Your work will directly impact the over 10 million patients who rely on Fullscript to stay connected to their care plans and follow through on treatment.

## What You Will Do - Lead the design, development, and deployment of production-ready, multi-turn LLM-powered features, including summarization tools and clinician-facing conversational agents.

• Own backend services in Python that integrate LLM agents with Fullscript's platform and support reliable production use.
• Help define technical direction for prompting, grounding, safety, and orchestration strategies used across clinical AI workflows.
• Establish and improve evaluation approaches for LLM outputs, including accuracy, hallucinations, edge cases, and overall feature quality.
• Shape engineering patterns for model-related workflows, including testing, CI/CD, observability, and version control.
• Partner with medical, product, and engineering teams to identify high-value opportunities for AI and turn them into practical, scalable product capabilities.
• Work cross-functionally with engineering, analytics, and medical SMEs to refine requirements and ensure data and system design support clinical use cases.
• Provide technical leadership across projects by creating clarity in ambiguous problem spaces, guiding tradeoff decisions, and raising the quality bar for the team.
• Stay current with the latest LLM research and emerging AI technologies, and help assess where they can be applied effectively at Fullscript.

## What We Are Looking For - 6+ years of experience building and implementing machine learning applications in production, with meaningful experience in LLM-powered agents, conversational experiences, or agent-based workflows.

• A track record of owning complex technical problems end-to-end and shaping implementation beyond immediate code contributions.
• Experience designing and deploying AI systems that answer open-ended questions, support follow-up interactions, and operate reliably in production.
• Strong experience with LLM application frameworks and tooling, such as LangChain, LangGraph, or similar orchestration and RAG frameworks.
• Familiarity with evaluation and monitoring frameworks for LLM outputs, conversational quality, and system reliability.
• Knowledge of MCP, agent orchestration patterns, or related approaches for building multi-step AI systems.
• Ability to work with a high degree of autonomy and collaborate effectively with cross-functional teams.
• Strong technical leadership and communication skills.

## Nice to Have - Experience with cloud-based AI services and platforms.

• Familiarity with healthcare-related data standards and regulations.
• Knowledge of software development methodologies, such as Agile.
• Experience with containerization using Docker and orchestration using Kubernetes.

## Benefits and Perks - Competitive compensation package.

• Opportunity to work with a cutting-edge healthcare technology company.
• Collaborative and dynamic work environment.
• Professional development opportunities.
• Flexible working hours and remote work options.
• Access to the latest technologies and tools.
• Comprehensive health insurance and benefits package.
• Generous PTO policy.

- Showcase your experience with LLM-powered agents and conversational experiences, highlighting any projects or applications you've developed.

• Emphasize your ability to work autonomously and collaboratively, as this role requires a high degree of self-motivation and teamwork.
• Be prepared to discuss your approach to evaluating and monitoring LLM outputs, including how you handle edge cases and ensure system reliability.
• Familiarize yourself with Fullscript's mission and values, and be ready to explain how your skills and experience align with the company's goals.
• When discussing your experience with machine learning frameworks and tooling, focus on specific examples and applications rather than just listing technologies.
• Highlight any experience you have working with cross-functional teams, including medical stakeholders, and how you've ensured that technical solutions meet clinical needs.