Databricks
WFA Digital Insight
The demand for skilled data engineers with expertise in Databricks has surged in recent years, driven by the exponential growth of big data and the need for scalable analytics solutions. With the global data engineering market expected to reach $77.37 billion by 2023, professionals with expertise in Databricks, AWS, and Azure are in high demand. PradeepIT Consulting Services Pvt Ltd is at the forefront of this trend, and this role offers a unique opportunity to work with cutting-edge technologies and collaborate with a talented team of experts. As the job market continues to evolve, candidates with a strong foundation in data engineering, analytics, and cloud computing will be well-positioned for success. Before applying, candidates should be prepared to showcase their technical skills, problem-solving abilities, and experience working with distributed systems.
Job Description
About the Role
As a Databricks expert at PradeepIT Consulting Services Pvt Ltd, you will play a critical role in developing modern data warehouse solutions using Databricks, AWS, and Azure. You will work closely with the DW/BI leads to understand new ETL pipeline development requirements and collaborate with cross-functional teams to design and implement scalable data pipelines. Your expertise in data engineering and analytics will enable the company to make data-driven decisions and drive business growth.The role involves working in a fast-paced environment with multiple concurrent projects, requiring strong analytical and problem-solving skills, as well as excellent communication and collaboration abilities. You will be responsible for driving technical discussions with client architects and team members, orchestrating data pipelines, and ensuring seamless data integration.
The company is committed to innovation and is looking for a talented professional who can bring new ideas and perspectives to the table. If you are passionate about data engineering, analytics, and cloud computing, and are looking for a challenging role that offers opportunities for growth and development, this could be the ideal fit.
What You Will Do
- Develop modern data warehouse solutions using Databricks, AWS, and Azure
- Collaborate with DW/BI leads to understand new ETL pipeline development requirements
- Design and implement scalable data pipelines using Databricks, AWS, and Azure
- Work with businesses to understand reporting needs and develop data models to fulfill those needs
- Resolve technical issues and challenges faced by team members
- Drive technical discussions with client architects and team members
- Orchestrate data pipelines using Airflow
- Collaborate with cross-functional teams to ensure seamless data integration
- Develop and maintain documentation of data pipelines and architectures
- Stay up-to-date with the latest trends and technologies in data engineering and analytics
What We Are Looking For
- Bachelor's or master's degree in computer science or equivalent experience
- At least 6 years of IT experience and 3 years of experience in data warehouse/ETL projects
- Strong understanding of Star and Snowflake dimensional modeling
- Experience with Databricks Data & AI platform and Databricks Delta Lake Architecture
- Hands-on experience with SQL, Python, and Spark (PySpark)
- Experience with AWS or Azure stack
- Strong knowledge of data management principles
- Experience with ETL tools and technologies (e.g., Informatica, Talend)
- Experience with data streaming technologies (e.g., Kafka, Kinesis)
Nice to Have
- Experience with Apache Kafka for streaming data/event-based data
- Experience with other open-source big data products (e.g., Hadoop, Hive, Pig, Impala)
- Experience with NoSQL data repositories (e.g., MongoDB, Cassandra, Neo4J)
- Experience working with structured and unstructured data (e.g., imaging, geospatial data)
- Experience working in a Dev/Ops environment with tools like Terraform, CircleCI, and GIT
Benefits and Perks
- Competitive salary and benefits package
- Opportunity to work with cutting-edge technologies and collaborate with a talented team of experts
- Flexible working hours and remote work options
- Professional development and training opportunities
- Access to a comprehensive library of online courses and tutorials
- Recognition and reward for outstanding performance and contributions
- A dynamic and innovative work environment with a strong focus on teamwork and collaboration
How to Stand Out
- Tip: Make sure your resume and online profiles (e.g., LinkedIn) are up-to-date and highlight your technical skills and experience with Databricks, AWS, and Azure.
- Tip: Prepare to showcase your problem-solving abilities and experience working with distributed systems during the interview process.
- Tip: Research the company's products and services, and be prepared to ask informed questions during the interview.
- Tip: Emphasize your experience working in a fast-paced environment with multiple concurrent projects, and highlight your ability to prioritize tasks and manage your time effectively.
- Tip: Be prepared to provide specific examples of your experience with data engineering and analytics, and explain how you have applied your skills and knowledge in previous roles.
- Tip: Consider obtaining Databricks Certified Data Engineer Associate/Professional Certification to demonstrate your expertise and commitment to the field.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.