Staff Software Engineer, AI

## About the Role The Staff Software Engineer, AI at Lattice plays a pivotal role in shaping the foundations of AI quality, reliability, and impact across the platform. This role is about architecting and scaling the infrastructure that powers AI quality, reliability, and reuse. As part of the AI Engineering team, you will work closely with cross-functional teams to define how intelligence works across the platform.

Day-to-day, you will be responsible for designing and scaling an end-to-end AI evaluation framework, architecting reusable agent infrastructure, and making principled build-vs-buy decisions. Your expertise in AI evaluation methodology and statistical experimentation will be crucial in identifying and quantifying the drivers of agent quality improvement.

## What You Will Do - Design and scale an end-to-end AI evaluation framework spanning offline evaluations, production tracing, and human feedback loops.

• Define meaningful performance metrics and build the datasets and automated scoring systems that prevent regressions.
• Identify and quantify the drivers of agent quality improvement and set methodological standards for evaluation across the organization.
• Architect reusable agent infrastructure using LangGraph or comparable frameworks.
• Build and scale RAG pipelines, vector retrieval systems, and production-grade AI infrastructure.
• Make principled build-vs-buy decisions across LLM providers, agent frameworks, and evaluation tooling.
• Engineer AI systems as reusable internal platforms that multiply product engineering velocity at Lattice.
• Own projects end-to-end: scope, design, execution, and delivery.
• Set technical direction for agent quality and evaluation strategy across Lattice engineering teams.
• Lead rigorous discussions on AI system design and evaluation methodology.
• Raise the AI engineering bar through mentorship, code review, and clear technical communication.

• Deep production experience with LLM systems and evaluation frameworks.
• Experience building and operating agentic systems and managing their failure modes.
• Strong command of AI evaluation methodology and statistical experimentation.
• Production-grade Python skills.
• Experience with LangGraph or comparable agent orchestration frameworks.
• Familiarity with vector databases and retrieval system design.
• Experience operating AI systems in cloud environments, including CI/CD, monitoring, and deployment workflows.

• Background in traditional ML and judgment in selecting ML vs. LLM approaches.
• Experience with MLOps tooling.
• Published research, talks, or open-source contributions in AI/ML.
• Experience in HR tech or other trust-sensitive domains.

• Opportunity to work remotely in Canada.
• Chance to be part of a cutting-edge team shaping the future of AI.
• Comprehensive health benefits.
• Generous PTO policy.
• Professional development opportunities.
• Access to the latest tools and technologies in AI and ML.

- Ensure you have a strong portfolio that showcases your experience in AI evaluation, agent architecture, and technical leadership.

• Prepare to discuss specific examples of how you've improved AI system quality and reliability in previous roles.
• Familiarize yourself with Lattice's technology stack, including LangGraph and comparable frameworks.
• Be ready to discuss your approach to statistical experimentation and AI evaluation methodology.
• Highlight your experience working in cloud environments and with CI/CD pipelines.
• Show enthusiasm for staying updated with the latest trends and research in AI and ML.
• Consider how your experience can contribute to Lattice's mission and vision, and be prepared to discuss this during the interview process.