AI Product Engineer - ClickStack
WFA Digital Insight
The demand for AI engineers with expertise in observability and real-time analytics has surged in recent years, with over 250 percent growth in some sectors. As companies like ClickHouse continue to innovate and lead the market, the need for skilled professionals who can build and integrate AI-powered solutions has never been more pressing. With the rise of remote work, candidates now have more opportunities to join cutting-edge companies like ClickHouse, which has been recognized as one of the most innovative and fast-growing private cloud companies. Before applying, candidates should be aware of the company's focus on real-time analytics, data warehousing, and AI workloads, as well as its commitment to open-source solutions.
Job Description
About the Role
ClickHouse is seeking an experienced AI Product Engineer to join its team in building the AI layer for its observability platform, ClickStack. As a key member of the team, you will be responsible for designing and developing agentic capabilities that can investigate incidents, surface anomalies, and provide root cause analysis. Your work will have a direct impact on the company's mission to transform how companies use data.The role entails working closely with the engineering team to build a library of reusable skills that capture the team's debugging processes, ClickHouse query writing, and incident response procedures. You will also own the agent stack end-to-end, ensuring that the system is scalable, reliable, and efficient. With a focus on developer experience, you will work to make ClickStack a great place to run AI workloads, building MCP servers, SDKs, and integrations that enable customers' agents to read telemetry, take action, and remain observable.
What You Will Do
- Build agents that investigate incidents and surface anomalies, using ClickStack as their substrate
- Design and develop reusable skills that capture the team's debugging processes and incident response procedures
- Own the agent stack end-to-end, ensuring scalability, reliability, and efficiency
- Collaborate with the engineering team to build MCP servers, SDKs, and integrations that enable customers' agents to read telemetry, take action, and remain observable
- Work in the open, collaborating with OSS contributors and customers to debug problems and feed learnings back into the product
- Tackle hard problems such as latency, cost, context window limits, eval coverage, and hallucinations on real telemetry
- Write skills, not just prompts, to build a library of reusable capabilities
- Build the agent layer, including systems that can investigate incidents at 2 AM, propose root causes, and provide concise summaries
- Ensure that the agent works in production, taking ownership of context engineering, tool design, evals, tracing, and cost
What We Are Looking For
- 5+ years of software engineering experience, including 1-2 years on LLM-powered systems or agents in production
- Strong backend skills in TypeScript/Node.js and/or Python, with comfort in both languages
- Hands-on experience building agents, including multi-step tool use, planning, memory, and error recovery
- Experience designing skills, such as Markdown-based workflow encodings or Anthropic-style skills
- Strong understanding of production terms, including p99 latency, cost per task, and system reliability
- Ability to move quickly, ship often, and learn from what breaks
- Passion for developer tools and a clear sense of what good DX looks like
- Ability to work with ambiguity and ownership
Nice to Have
- Experience with real-time analytics, data warehousing, and AI workloads
- Familiarity with open-source solutions and collaborative development
- Knowledge of containerization, orchestration, and cloud-native technologies
- Experience with CI/CD pipelines and automated testing
Benefits and Perks
- Opportunity to work with a cutting-edge company and technology
- Collaborative and dynamic work environment
- Flexible and remote work arrangements
- Access to professional development and growth opportunities
- Competitive compensation and benefits package
- Equity and stock options
- Generous PTO and paid holidays
- Remote stipend and home office setup support
How to Stand Out
- Tip: Make sure to highlight your experience with LLM-powered systems and agents in production, as this is a key requirement for the role.
- Familiarize yourself with ClickHouse's technology stack and be prepared to discuss your experience with similar technologies.
- Showcase your passion for developer tools and your understanding of what good DX looks like.
- Be prepared to provide examples of your work, including code repositories or personal projects that demonstrate your skills.
- Tip: Research the company's culture and values, and be prepared to discuss how you align with them.
- When negotiating salary, be sure to consider the company's equity and stock options, as well as other benefits and perks.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.