Principal Software Engineer - Data Lakes

About the Role

The modern data stack is evolving rapidly, with more companies turning to cloud-based data lakes for their data management needs. Fivetran is well-positioned to capitalize on this trend, and your contributions as a Principal Software Engineer will be key to driving this growth. You will be working remotely from Germany, EMEA, as part of a senior engineering team, to ensure the success of Fivetran's data lake initiatives.

Fivetran's mission is to make access to data as simple and reliable as electricity. With thousands of customers relying on Fivetran's technology to become truly data-driven, the stakes are high, but the potential for impact is immense. As a Principal Software Engineer, you will be instrumental in shaping the technical direction of the company's data lake offerings and ensuring that they meet the highest standards of reliability, performance, and security.

What You Will Do

• Partner closely with product teams to understand requirements and design cutting-edge new capabilities that go directly into customers' hands.
• Design, develop, implement, and operate highly reliable large-scale data lake systems in cooperation with a dedicated data lake engineering team.
• Contribute to open-source projects such as DuckDB and Polaris to advance Fivetran's technology and reputation in the industry.
• Embrace Fivetran innovations with open-source standards and toolsets to ensure compatibility and future-proofing.
• Analyze fault-tolerance and high availability issues, performance and scale challenges, and solve them through rigorous engineering and problem-solving.
• Ensure operational excellence of the services and meet the commitments to customers regarding security, reliability, availability, and performance.
• Set technical directions and influence cross-functional teams to drive alignment and excellence across the organization.
• Participate in the development of the company's technical roadmap, ensuring that data lake systems are aligned with business objectives and customer needs.
• Collaborate with the engineering team to identify areas for improvement and implement processes that enhance efficiency, quality, and reliability.

What We Are Looking For

• 6+ years of hands-on or research experience with high-performance relational data management systems.
• Deep understanding of infrastructure & software optimizations and performance engineering to drive significant performance, latency, and availability improvements.
• Proven track record of leading and delivering large and complicated projects, with a focus on data lakes and cloud-based technologies.
• Strong development skills in Java and C++, with experience in designing and developing scalable systems.
• Solid experience with public clouds (AWS, Azure, GCP), including migration, deployment, and management of cloud-based data lake systems.
• Demonstrated knowledge of columnar storage formats, data warehousing, and big data technologies.
• Experience with containerization (e.g., Docker) and orchestration (e.g., Kubernetes) for deploying and managing data lake systems.
• Familiarity with agile development methodologies and version control systems (e.g., Git).

Nice to Have

• Experience with data lake technologies such as Iceberg, Polaris, Delta Lake, Parquet, and DuckDB.
• Knowledge of gRPC and Temporal for building scalable and reliable data processing pipelines.
• Familiarity with Grafana for monitoring and visualizing data lake performance and health.
• Experience contributing to open-source projects, particularly in the data management and cloud computing space.
• Certification in cloud computing (e.g., AWS Certified Developer) or data engineering (e.g., Google Cloud Certified - Professional Data Engineer).

Benefits and Perks

• Competitive salary and equity package.
• Opportunity to work on cutting-edge data lake technologies and contribute to open-source projects.
• Collaborative and dynamic work environment with a team of experienced engineers.
• Flexible working hours and remote work arrangement to ensure work-life balance.
• Professional development opportunities, including training, mentorship, and conference attendance.
• Access to the latest tools and technologies for data lake development and management.
• Comprehensive health insurance and retirement plans.
• Generous paid time off and vacation policy.

• Develop a strong understanding of cloud-based data lakes: Familiarize yourself with the latest technologies and trends in data lake management, including cloud providers like AWS, Azure, and GCP.
• Highlight your experience with open-source projects: If you have contributed to open-source projects like DuckDB or Polaris, make sure to highlight this in your application, as it demonstrates your commitment to the field and your ability to collaborate with others.
• Showcase your problem-solving skills: Provide specific examples of how you have analyzed and solved complex technical problems in your previous roles, particularly those related to data lake systems and cloud computing.
• Prepare to discuss your experience with public clouds: Be ready to talk about your experience with AWS, Azure, or GCP, including deployment, management, and optimization of cloud-based data lake systems.
• Emphasize your ability to work remotely: As this is a remote position, highlight your experience working remotely and your ability to manage your time effectively, communicate clearly with team members, and maintain productivity in a remote work environment.
• Research Fivetran's technology and mission: Show your interest in the company by researching Fivetran's products, mission, and values, and be prepared to discuss how your skills and experience align with these.
• Be prepared to back up your claims: Make sure you can provide concrete examples to support your claims of expertise and experience, particularly in areas like data lake development, cloud computing, and open-source contribution.