Big Data Engineer IV
WFA Digital Insight
The demand for skilled big data engineers has skyrocketed in recent years, with the global big data market projected to reach $243 billion by 2027. As a Big Data Engineer IV at OpenX, you'll play a critical role in designing and implementing large-scale data processing systems, working with a talented team to drive innovation in digital advertising. With the industry experiencing rapid growth, professionals with expertise in Java, Scala, or Python are in high demand. Before applying, candidates should be prepared to showcase their technical skills and experience in big data systems development, as well as their ability to work collaboratively in a remote environment.
Job Description
About the Role
As a Big Data Engineer IV at OpenX, you will be responsible for designing, implementing, and maintaining large-scale data processing systems that can handle billions of requests per day. You will work closely with the product team to drive requirements and own projects end-to-end, analyzing and improving the efficiency, scalability, and stability of applications. Your expertise in Java, Scala, or Python will be essential in developing complex data pipelines, including SQL-based transformations and large-scale data processing with distributed frameworks such as Apache Spark.The Big Data Engineer IV role is part of the core data platform team, which handles billions of ad requests daily, connecting thousands of publishers with demand partners. Each request produces data events that need to be processed to extract business value, resulting in over 1PB of data produced daily. You will be working with multiple distributed teams to architect, create, and deliver new features and functionality, focusing on scalability, performance, and reliability.
What You Will Do
- Design large-scale data processing systems to handle billions of requests per day
- Work with the product team to drive requirements and own projects end-to-end
- Analyze and improve the efficiency, scalability, and stability of applications
- Develop complex data pipelines, including SQL-based transformations and large-scale data processing with distributed frameworks
- Collaborate with multiple distributed teams to architect, create, and deliver new features and functionality
- Focus on scalability, performance, and reliability in every aspect of the system
- Identify and remove unnecessary complexity in existing systems
- Think long-term and be unsatisfied with temporary fixes
- Troubleshoot operational issues and extend data pipelines as needed
- Work with data orchestration tools like Airflow to manage and monitor data workflows
What We Are Looking For
- 8+ years of experience in Java, Scala, or Python development
- 3+ years of experience in Big Data systems development
- Experience building complex data pipelines, including SQL-based transformations and large-scale data processing
- Expertise using BigQuery or other data warehouses, including performance tuning and monitoring
- Understanding of data orchestration tools like Airflow
- Cloud experience with Google Cloud Platform or AWS, k8s, and Docker
- Understanding of CI/CD, automated testing, monitoring, and other DevOps practices
- Fluency in algorithms, data structures, and platforms
- Experience with database system design, RDBMs, and/or NoSQL
- Bachelor's degree in computer science, physics, or a related field
- A Master's degree is a plus
- Communicative Polish and English
Nice to Have
- Experience working in digital media, marketing technology, or advertising technology
- SCRUM/Agile environment experience
- Ability to develop scalable, modular applications
- Experience with Excel for data analysis and reporting
- Good written and oral communication skills
- Good sense of humor and a team player mentality
Benefits and Perks
- Employment contract or B2B options available
- Competitive salary (gross/monthly or net/hourly)
- Opportunities to work with the newest technologies, such as Cloud Computing (GCP)
- Challenges at work that are difficult to find elsewhere
- Solving important problems in a scale that makes a difference
- Joining a company that is growing and scaling rapidly
- Flexible working hours and remote work options
- Annual performance bonus
- Tax-deductible system due to copyright protection
- Private health care for you and your family
- Other benefits typical of remote jobs, including home office stipend and professional development opportunities
How to Stand Out
- Tip: Showcase your expertise in Java, Scala, or Python, and highlight your experience in Big Data systems development.
- Familiarize yourself with data orchestration tools like Airflow and be prepared to discuss how you've used them in previous roles.
- Make sure your resume and cover letter are tailored to the Big Data Engineer IV role, emphasizing your relevant technical skills and experience.
- Be prepared to discuss your experience with database system design, RDBMs, and/or NoSQL, and how you've applied these skills in previous positions.
- Research OpenX and the digital advertising industry to understand the company's role and how you can contribute to its mission.
- Be prepared to provide specific examples of times when you had to troubleshoot operational issues or optimize system performance.
- Highlight your ability to work collaboratively in a remote environment and your communication skills, both written and verbal.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.