Principal Engineer, Streaming Systems
WFA Digital Insight
As the demand for real-time data processing and AI-driven marketing solutions continues to soar, with the global market expected to reach
Job Description
About the Role
The Principal Engineer position at Hightouch is a unique opportunity for a highly skilled engineer to lead the development of high-throughput Streaming Systems. These systems are crucial for collecting, forwarding, and performing real-time computation on vast amounts of data, driving the core functionality of Hightouch's AI marketing platform. The ideal candidate will have a deep understanding of distributed systems, a passion for solving complex technical problems, and the ability to architect and deploy scalable solutions.The role involves working closely with cross-functional teams to ensure seamless integration of the Streaming Systems with other components of the platform. This includes collaborating with product managers to understand requirements, engineers to implement solutions, and DevOps teams to ensure smooth deployments. The Principal Engineer will also be responsible for mentoring junior engineers, fostering a culture of technical excellence, and contributing to the company's technical roadmap.
What You Will Do
- Design and implement high-throughput data pipelines capable of handling millions of messages per second with sub-second latency.
- Lead the development of real-time computation engines that personalize user experiences based on live behavioral data.
- Collaborate with the product team to identify key performance indicators (KPIs) for the Streaming Systems and develop dashboards for monitoring and optimization.
- Develop and maintain large-scale data processing architectures, ensuring fault tolerance, scalability, and high availability.
- Implement caching strategies to improve the performance of online computation services.
- Architect systems for deduplicating message delivery at high throughputs.
- Develop patterns for zero-downtime deployments in online computation services.
- Work with the engineering team to ensure that all systems are properly documented and that knowledge is shared across the team.
- Participate in the on-call rotation, providing 24/7 support for critical systems.
What We Are Looking For
- A bachelor's or master's degree in Computer Science, Electrical Engineering, or a related field.
- 8+ years of experience in software development, with a focus on distributed systems, data processing, or related fields.
- Proven experience in leading teams and mentoring junior engineers.
- Strong understanding of computer science fundamentals, including data structures, algorithms, and software design patterns.
- Experience with cloud-based technologies, such as AWS or GCP, and containerization using Docker.
- Familiarity with data processing frameworks like Apache Kafka, Apache Storm, or similar technologies.
- Excellent communication and interpersonal skills, with the ability to work effectively in a distributed team environment.
- Adept at handling multiple projects simultaneously and prioritizing tasks to meet deadlines.
- Experience with Excel for data analysis and reporting.
Nice to Have
- Experience with machine learning frameworks and AI technologies.
- Knowledge of container orchestration using Kubernetes.
- Familiarity with agile development methodologies and version control systems like Git.
- Experience with monitoring and logging tools such as Prometheus, Grafana, and ELK Stack.
Benefits and Perks
- Competitive salary and equity package.
- Opportunity to work on cutting-edge technologies and contribute to the development of AI-driven marketing solutions.
- Collaborative and dynamic work environment with a team of highly skilled professionals.
- Flexible working hours and remote work options.
- Professional development opportunities, including training, mentorship, and conference attendance.
- Access to the latest tools and technologies.
- Comprehensive health insurance and retirement plans.
- Generous parental leave policy and family support benefits.
How to Stand Out
- Ensure your resume and cover letter highlight your experience with distributed systems, data processing, and leadership roles.
- Prepare to discuss specific examples of scalability challenges you've faced and how you overcame them.
- Brush up on your knowledge of real-time data processing technologies and cloud computing platforms.
- Develop a portfolio that showcases your technical skills and accomplishments, especially in areas relevant to the role.
- Research the company culture and be ready to discuss how your values and work style align with Hightouch's mission and principles.
- Don't hesitate to ask about the team's dynamics, the company's approach to professional development, and what a typical day in this role looks like during the interview.
- Be prepared to negotiate your salary based on your research of the market standards for similar positions, and don't forget to inquire about the equity package and its vesting schedule.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.