Software Engineer - Database Integrations

ClickhouseClickhouse·Remote(Germany)
Software Development
Adjust

WFA Digital Insight

The demand for skilled software engineers in database integrations has grown exponentially, with the global cloud market projected to reach

.5 trillion by 2027. ClickHouse, a leader in real-time analytics and data warehousing, is at the forefront of this trend. As a software engineer at ClickHouse, you'll be working on complex database-related challenges, leveraging your expertise in systems programming, cloud-native technologies, and Kubernetes. With the company's rapid growth and innovative approach, this role offers a unique opportunity to work on cutting-edge projects and collaborate with a talented team. Before applying, candidates should be prepared to showcase their problem-solving skills, experience with database replication technologies, and ability to work in a fast-paced, remote environment.

Job Description

About the Role

As a Software Engineer at ClickHouse, you will be part of the Database Integrations team, responsible for building the platform that enables real-time data replication from databases into ClickHouse at petabyte scale. This role entails solving complex database-related challenges and distributed systems problems, such as optimizing snapshotting strategies, handling schema evolution, and ensuring data consistency. You will work closely with the team to design and develop high-throughput integrations with databases, data lakes, and data warehouses.

The ClickPipes team is committed to working in the open, with database integrations built on PeerDB, an open-source CDC platform. Recent projects include ClickPipes for Postgres, MongoDB CDC to ClickHouse, and MySQL Change Data Capture. As a member of this team, you will have the opportunity to contribute to these projects and drive innovation in the field of database integrations.

What You Will Do

  • Design and develop high-throughput integrations with databases (Postgres, MySQL, MongoDB), data lakes (Iceberg, Delta Lake), and data warehouses (BigQuery, Snowflake)
  • Handle edge cases in real-world production scenarios, such as unconventional database setups and large transactions
  • Optimize database internals to improve snapshotting strategy and reduce latency
  • Develop integration solutions to enable users to fully harness ClickHouse's performance and throughput
  • Own end-to-end reliability and debug complex issues in production using runtime diagnostics and observability tools
  • Build and improve infrastructure and tools to increase system reliability and reduce incident response time
  • Write clear documentation, both publicly and internally, to facilitate knowledge sharing and collaboration
  • Participate in on-call rotation to ensure 24/7 support for production systems
  • Drive product innovation by working directly with customers to understand integration requirements and discover gaps in existing products

What We Are Looking For

  • 5+ years of industry experience building data-intensive software solutions
  • Proficiency in Go, or experience in systems programming with a willingness to ramp up quickly in Go
  • Cloud-native experience deploying and operating services on at least one major cloud platform (AWS/GCP/Azure)
  • Practical experience with Kubernetes and containerization
  • Strong problem-solving skills and solid production debugging skills
  • Clear communication in writing (design docs, code review) and verbally (technical discussions, customer calls, incident response)
  • Experience with database replication technologies (CDC, logical replication) and durable execution frameworks (Temporal)
  • Familiarity with data formats and protocols (Avro, Parquet, Protobuf) and modern data processing frameworks (e.g., Kafka, Spark, Flink)

Nice to Have

  • Experience with maintaining or contributing to open-source software
  • Familiarity with modern data processing frameworks (e.g., Kafka, Spark, Flink)
  • Knowledge of data warehousing and business intelligence concepts
  • Experience with cloud security and compliance

Benefits and Perks

  • Competitive salary and benefits package
  • Opportunity to work on cutting-edge projects and collaborate with a talented team
  • Flexible working hours and remote work options
  • Professional development opportunities, including training and conference sponsorships
  • Access to the latest technologies and tools
  • Recognition and reward for outstanding performance

How to Stand Out

  • Tip: Highlight your experience with cloud-native technologies, such as Kubernetes and containerization, to stand out as a candidate.
  • Tip: Showcase your problem-solving skills by providing examples of complex database-related challenges you've solved in previous roles.
  • Tip: Familiarize yourself with ClickHouse's products and services, including PeerDB and ClickPipes, to demonstrate your enthusiasm for the company and role.
  • Tip: Prepare to discuss your experience with database replication technologies, such as CDC and logical replication, and durable execution frameworks like Temporal.
  • Tip: Emphasize your ability to work in a fast-paced, remote environment and your experience with collaborative development tools, such as Git and code review platforms.
  • Tip: Be prepared to provide examples of your writing skills, such as design documents or code reviews, to demonstrate your ability to communicate complex technical concepts clearly and concisely.

This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.