Staff Software Engineer Data - DC Tech Lead
WFA Digital Insight
The demand for skilled data engineers has skyrocketed, with a 25% increase in job openings in the past year alone. As companies like Afresh continue to innovate and scale, the need for experts in data integration and architecture has become crucial. With Afresh's mission to eliminate food waste, this role offers a unique opportunity to make a tangible impact. Candidates should be prepared to showcase their technical skills, particularly in PySpark and DBT, as well as their ability to collaborate and lead cross-functional teams.
Job Description
About the Role
As a Staff Software Engineer Data at Afresh, you will be part of the Distribution Center (DC) Solutions team, responsible for building and scaling the data integrations necessary to support the company's suite of DC products. Your day-to-day work will involve designing and implementing ETLs that reliably process large volumes of customer-provided data, as well as developing tools to improve the platform and make customer integrations faster, more accurate, and more scalable.The success of Afresh's products and services relies heavily on the ability to collect, process, and analyze vast amounts of data. As a technical lead, you will play a critical role in ensuring that the data infrastructure is robust, efficient, and capable of handling the demands of a rapidly growing customer base.
The DC Solutions team is a collaborative and dynamic group, working closely with cross-functional teams including product, engineering, and applied science. As a Staff Software Engineer, you will have the opportunity to mentor and guide junior engineers, contribute to the technical vision of the team, and drive the development of innovative solutions to real-world problems.
What You Will Do
- Design, build, and optimize robust ETLs using PySpark and DBT to process large-scale customer datasets
- Develop tools and frameworks to streamline data integrations and improve scalability
- Collaborate with the product team to scope work and deliver data solutions that address real-world challenges in customer data quality and product feature requirements
- Mentor and guide junior engineers to help them grow in their roles
- Define the technical vision for DC data architecture and ensure the team delivers high-quality, practical solutions
- Manage external contractors and ensure effective communication and collaboration
- Partner with the applied science team to develop and implement machine learning models that drive business value
- Identify areas where tooling or automation can simplify workflows and reduce manual effort
- Develop and maintain technical documentation to ensure knowledge sharing and continuity
- Stay up-to-date with industry trends and emerging technologies, applying this knowledge to drive innovation and improvement
What We Are Looking For
- Significant experience designing and maintaining ETLs that process large-scale datasets
- Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT
- 2+ years of experience in a technical lead role, with a willingness to mentor and help others grow
- Strong problem-solving skills and the ability to work with ambiguous or incomplete requirements
- A focus on practical outcomes and the ability to balance technical rigor with the need to get things done
- Experience working directly with complex, unclean datasets and finding innovative ways to process and analyze them
- Excellent communication skills, with the ability to explain technical concepts clearly to both technical and non-technical audiences
- A strong understanding of data architecture and the ability to design and implement scalable data systems
Nice to Have
- Experience with cloud-based data platforms, such as AWS or GCP
- Familiarity with Agile development methodologies and version control systems like Git
- Knowledge of data visualization tools and techniques, such as Tableau or Power BI
- Experience with machine learning or artificial intelligence applications
Benefits and Perks
- Comprehensive medical, dental, and vision coverage for you and your family
- Dedicated mental health support and counseling services
- Opportunities for professional growth and development, including training and education programs
- Flexible working hours and remote work options
- Access to cutting-edge technologies and tools
- Collaborative and dynamic work environment
- Recognition and reward for outstanding performance and contributions
- Comprehensive support system designed to help you do your best work while maintaining a healthy, balanced life
How to Stand Out
- Make sure to highlight your experience with PySpark and DBT, as well as your ability to design and implement scalable ETLs.
- Be prepared to discuss your approach to data architecture and your experience working with large-scale datasets.
- Showcasing your problem-solving skills and ability to work with ambiguous or incomplete requirements is crucial.
- Emphasize your ability to communicate technical concepts clearly and effectively to both technical and non-technical audiences.
- Be prepared to discuss your experience with machine learning or artificial intelligence applications, and how you can apply this knowledge to drive business value.
- When discussing your experience as a technical lead, focus on specific examples of how you have mentored and guided junior engineers, and how you have contributed to the technical vision of your team.
- Be prepared to ask questions about the company culture, the team you will be working with, and the opportunities for growth and development within the company.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.