Staff Software Engineer, Observability Platform
WFA Digital Insight
Wealthsimple’s Observability Platform team is looking for a senior engineer who will own the core telemetry stack that powers every production service. Unlike generic SRE or dev‑ops roles, this position treats the platform as a product, demanding deep expertise in event‑driven architectures, high‑cardinality storage, and OpenTelemetry standards. The engineer will design SDKs that make instrumentation the path of least resistance, and build query foundations that let AI agents troubleshoot incidents alongside humans. Candidates should be comfortable navigating unfamiliar codebases, mentoring senior peers, and shaping technical standards that affect the entire engineering organization. Success here means turning raw production data into actionable insight for both engineers and AI tools.
Job Description
BUILD SOMETHING PEOPLE LOVE Wealthsimple is Canada’s leading financial innovator. The company offers a full suite of simple, sophisticated financial products across managed investing, do-it-yourself trading, cryptocurrency, tax filing, spending and saving. Wealthsimple currently serves more than 4 million Canadians and holds over $155 billion in assets under administration. The company was founded in 2014 by a team of financial experts and technology entrepreneurs, and is headquartered in Toronto, Canada. We're proud of what we've built — and we're just getting started. Read our Culture Manual https://www.wealthsimple.com/en-ca/culture and learn more about how we work https://www.wealthsimple.com/en-ca/careers. ABOUT THE OBSERVABILITY PLATFORM TEAM The Observability Platform team exists to make production legible to every engineer and AI agent at Wealthsimple, on their first day and every day after. We treat the platform as a product, with our internal engineering teams as our customers. Our mission is to provide the best, most consistent logging, metrics, and tracing experience. We do this by brokering tools and patterns into golden paths that accelerate best practices, and by connecting teams to business impact through SLOs, user flows, and customer experience signals. We instrument on open standards so that switching costs are a configuration change rather than a rewrite, we optimize for the questions engineers did not anticipate rather than for pre-built dashboards, and we design for a world where both humans and AI agents investigate production through fast, high-cardinality data. As the Staff Software Engineer on this team, you are the deepest technical builder and the person who sets the technical bar. You design the foundations, write the code that matters most, and raise the engineering quality of everyone around you. IN THIS ROLE YOU'LL HAVE THE OPPORTUNITY TO: - Build the Event Foundation: Own the architecture for wide-event data models, high-cardinality storage, and high-throughput ingestion and query pipelines (columnar/streaming).
- Craft SDKs & Golden Paths: Design, ship, and drive adoption for telemetry libraries and SDKs that make rich instrumentation the path of least resistance.
- Set Technical Standards: Define instrumentation conventions, naming, tagging, sampling, and OpenTelemetry standards so humans and machines speak one language.
- Enable AI Agents in Production: Build fast query foundations and access patterns (e.g., Model Context Protocol / MCP) that allow AI agents to investigate incidents and verify changes alongside engineers.
- Build Fast with AI: Leverage modern AI tools (like Claude Code) to prototype, navigate complex systems, and help elevate the team’s AI-assisted workflows.
- Navigate Ambiguity: Dive into unfamiliar codebases to deliver significant, well-reasoned, high-impact changes safely.
- Experiment & Scale: Pilot new approaches with select teams, measure impact, and scale what works before committing broadly.
- Raise the Technical Bar: Mentor senior engineers, review complex architecture, and lead cross-team technical initiatives through influence.
- Multi-Language Coder: Current, hands-on experience writing production code in multiple languages (e.g., Kotlin, Ruby).
- Proven Telemetry Experience: Prior success building telemetry platforms, ingestion pipelines, and developer-facing SDKs.
- Event & Scale Mastery: Deep understanding of wide structured events, streaming/columnar backends, cardinality, sampling, and context propagation.
- Open Standards Expert: Hands-on experience with OpenTelemetry (or equivalent) and designing SLIs/SLOs.
- System Navigator: Ability to quickly map out, safely refactor, and impact complex, ambiguous codebases you didn't originally build.
- AI-Fluent Developer: Daily proficiency with AI tools (like Claude Code) to move faster while maintaining high quality standards.
- Strong Influencer: Exceptional communication skills to partner with cross-functional engineering teams and explain complex tradeoffs clearly.
- Hands-on experience with ClickHouse or columnar stores.
- Exposure to Kubernetes and progressive delivery tools (e.g., Argo Rollouts).
- Experience building observability solutions specifically for LLM and agent-based workloads. WHY WEALTHSIMPLE? 🌸 Top-tier health benefits and life insurance 📈 Long-term group savings with employer match, through Wealthsimple for Business 🌴 20 vacation days, 4 wellness days, and unlimited sick and mental health days per year ✈️ 90 days away: work outside Canada for up to 90 days per year 👥 Employee resource groups, including Rainbow (2SLGBTQ), Women of WS, and Black at WS 🌎 We are a hybrid team with over 1,500 employees across North America.
How to Stand Out
- Highlight any production‑grade telemetry SDKs you’ve built; include links to open‑source repos or code samples.
- Demonstrate familiarity with OpenTelemetry by mentioning concrete implementations or contributions.
- Prepare a short walkthrough of a high‑throughput ingestion pipeline you designed, focusing on scalability decisions.
- Show how you’ve used AI coding assistants in past projects; be ready to discuss concrete productivity gains.
- During interviews, ask about the team’s current golden paths and how they measure adoption across engineering squads.
- Emphasize experience mentoring senior engineers and influencing architectural standards.
- Be aware that Wealthsimple values product thinking for internal tools; frame your answers around user impact and developer experience.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.