Senior Software Engineer, AI

About the Role

The role requires a deep understanding of AI engineering, statistics, and software development. You will be responsible for designing and shipping a robust end-to-end AI evaluation framework, defining and instrumenting key metrics, and building and maintaining evaluation datasets and test harnesses. Your expertise in AI engineering will be crucial in shaping the company's AI products and driving innovation in the industry.

Lattice's engineering team is committed to continuous improvement and collaboration. As a Senior Software Engineer, AI, you will have the opportunity to work with a talented team of engineers, product managers, and designers to deliver high-quality AI products. You will be expected to contribute to the development of the company's technical direction, provide technical leadership, and collaborate with other teams to drive business outcomes.

What You Will Do

• Design and ship a robust, end-to-end AI evaluation framework, covering offline evals, production tracing, and human-in-the-loop feedback loops.
• Define and instrument key metrics, such as agent task completion, hallucination rates, response quality, user engagement, and downstream business outcomes.
• Build and maintain evaluation datasets, test harnesses, and automated scoring pipelines to catch regressions before they ship.
• Identify and surface the drivers of agent quality improvement, giving the team clear signals on where to invest.
• Architect and implement reusable agent infrastructure, including multi-turn conversation workflows, recommendation services, and standardized agent topology patterns.
• Build and scale RAG pipelines and retrieval infrastructure, including vector store management and retrieval quality optimization.
• Contribute to production AI systems with a strong focus on reliability, observability, and performance.
• Own projects end-to-end, driving them to completion and bringing in the right people at the right time.
• Partner with engineering leads and managers to inform technical direction on agent quality and evaluation strategy.
• Raise the AI engineering bar across the broader team through code review, documentation, and thoughtful technical debate.

What We Are Looking For

• 5+ years of professional software engineering experience with significant time spent on production AI/ML systems.
• Deep hands-on experience with LLM-based systems, including prompt engineering, RAG pipelines, agent orchestration, evaluation metrics, and model fine-tuning.
• Proven ability to work with data and understand statistics, especially in experiments.
• Proven ability to build and operate agentic AI systems in production, including multi-step workflows and multi-agent topologies.
• Strong command of AI evaluation, including building eval frameworks and understanding the difference between good evals and vanity metrics.
• Production-grade Python engineering, including clean, maintainable, testable code.
• Experience with LangGraph or comparable agent orchestration frameworks.
• Experience with LangSmith or comparable LLM observability tooling for tracing, evaluation, and debugging.

Nice to Have

• Experience with cloud-based AI platforms, such as AWS SageMaker or Google Cloud AI Platform.
• Knowledge of containerization, including Docker and Kubernetes.
• Familiarity with agile development methodologies, such as Scrum or Kanban.
• Experience with collaborative development tools, such as GitHub or GitLab.

Benefits and Perks

• Competitive salary and benefits package.
• Opportunity to work with a talented team of engineers, product managers, and designers.
• Collaborative and dynamic work environment.
• Flexible working hours and remote work options.
• Professional development opportunities, including training and conference attendance.
• Access to cutting-edge AI technologies and tools.
• Recognition and reward for outstanding performance.
• Comprehensive health and wellness programs.

• Tip: Make sure your resume and online profiles highlight your experience with AI engineering and production AI/ML systems.
• Be prepared to discuss your understanding of AI evaluation frameworks and your ability to design and implement robust AI systems.
• Showcasing your ability to work collaboratively in a remote environment is crucial, so be ready to provide examples of successful remote projects.
• Familiarize yourself with Lattice's products and services, and be prepared to discuss how your skills and experience align with the company's goals.
• Practice your coding skills, especially in Python, and be prepared to complete coding challenges or whiteboarding exercises during the interview process.
• Research the company culture and values, and be prepared to discuss how you can contribute to and thrive in Lattice's dynamic and collaborative environment.
• Consider creating a portfolio that showcases your AI engineering projects and accomplishments, and be prepared to discuss your design decisions and technical choices.