Staff Software Engineer, Data Infrastructure

Instacart·Remote(United States)
Software Development
Excel

WFA Digital Insight

The demand for skilled data infrastructure engineers has grown significantly, with a 25% increase in job postings over the last year. As companies like Instacart continue to scale, the need for professionals who can balance technical complexity with business acumen is more pressing than ever. With its flexible approach to work and commitment to innovation, Instacart stands out as an attractive option for those looking to make a meaningful impact in the industry. Before applying, candidates should be prepared to showcase their expertise in data lakehouses, streaming infrastructure, and AI-native data engineering, as well as their ability to thrive in a fast-paced environment where technical decisions have real business consequences.

Job Description

About the Role

Instacart is seeking a highly skilled Staff Software Engineer to join its Data Infrastructure organization. As a key member of the Data Governance and Foundations Team, you will play a critical role in shaping the company's data strategy and driving the development of its open lakehouse foundation. You will work closely with engineering leadership and stakeholders across various teams, including Data Science, ML Platform, Ads Infrastructure, Finance Engineering, Product Engineering, and Security.

The Data Infrastructure organization is responsible for building and operating the systems that power Instacart's data ecosystem. This includes a modern open data lakehouse on Apache Iceberg, a multi-engine compute platform for stream and analytical workloads, and self-serve tooling that enables Product, Data Science, ML, Ads, Finance, and engineering teams to move quickly with data. As a Staff Software Engineer, you will have the opportunity to work on complex, high-scale challenges and contribute to the development of Instacart's data platform.

Instacart's flexible approach to work allows employees to choose where they do their best work, whether that's from home, an office, or a favorite coffee shop. The company values pragmatism, clarity, and impact, and is looking for individuals who share these values and are passionate about delivering high-quality results.

What You Will Do

  • Translate Instacart's data strategy into an actionable multi-year architecture roadmap, aligning with leadership and evolving the platform for scale, maturity, and cost efficiency
  • Own the open lakehouse foundation, defining and delivering unified table formats, storage governance, and a multi-engine compute portfolio that enables portability and prevents lock-in
  • Drive real-time and streaming infrastructure for critical use cases, such as Ads and Fraud, setting deployment patterns, SLAs, and operational practices that balance performance, availability, and spend
  • Pioneer AI-native data infrastructure engineering by applying LLM/AI tools to the platform lifecycle, accelerating development, automation, observability, and cost optimization
  • Elevate engineering excellence, leading architecture reviews, mentoring senior/staff engineers, influencing hiring, and clearly communicating complex trade-offs to technical and executive audiences
  • Collaborate with stakeholders across various teams to ensure cross-org alignment and drive the development of Instacart's data platform
  • Develop and maintain technical documentation, ensuring that all systems and processes are well-documented and easily accessible
  • Participate in on-call rotations, providing 24/7 support for Instacart's data infrastructure
  • Stay up-to-date with industry trends and emerging technologies, applying this knowledge to continuously improve Instacart's data platform

What We Are Looking For

  • 5+ years of software engineering experience building and operating data infrastructure or distributed systems at production scale
  • Hands-on expertise with modern data lakehouses, including Apache Iceberg and Delta Lake
  • Experience with stream and analytical workloads, including Apache Flink, Trino, and Apache Spark
  • Strong understanding of data governance and access patterns, including storage governance and multi-engine compute portfolios
  • Experience with AI-native data infrastructure engineering, including the application of LLM/AI tools to platform development
  • Strong technical leadership skills, with experience leading architecture reviews and mentoring senior/staff engineers
  • Excellent communication skills, with the ability to clearly communicate complex technical concepts to technical and executive audiences
  • Experience with cloud-based technologies, including AWS and Snowflake
  • Strong understanding of data security and compliance, including experience with data encryption and access controls

Nice to Have

  • Experience with Confluent, Airflow, and dbt
  • Strong understanding of machine learning and data science concepts, including experience with ML platforms and tools
  • Experience with Agile development methodologies and version control systems, including Git
  • Strong understanding of DevOps principles and practices, including experience with containerization and orchestration
  • Experience with data visualization tools, including Tableau and Power BI

Benefits and Perks

  • Competitive salary and equity package
  • Flexible work arrangements, including remote work options and compressed workweeks
  • Comprehensive health, dental, and vision insurance
  • 401(k) matching and retirement savings plan
  • Generous paid time off and holidays
  • Access to cutting-edge technologies and tools
  • Opportunities for professional development and growth
  • Collaborative and dynamic work environment

How to Stand Out

  • Develop a strong understanding of data lakehouses and streaming infrastructure, including Apache Iceberg and Apache Flink
  • Showcase your expertise in AI-native data infrastructure engineering, including the application of LLM/AI tools to platform development
  • Highlight your experience with data governance and access patterns, including storage governance and multi-engine compute portfolios
  • Prepare to discuss your approach to technical leadership and architecture reviews, including your experience mentoring senior/staff engineers
  • Be ready to explain your experience with cloud-based technologies, including AWS and Snowflake
  • Emphasize your understanding of data security and compliance, including experience with data encryption and access controls
  • Consider creating a portfolio of your work, including examples of your experience with data infrastructure and AI-native engineering

This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.