Cloud Data Engineer - ETL/Data Warehousing (5-8 yrs)

Key Responsibilities :

- Design, develop, and implement data pipelines using Azure Data Factory, Databricks, and other Azure data services.

- Develop and maintain ETL/ELT processes for data ingestion, transformation, and loading into data warehouses and data lakes.

- Work with stakeholders to understand business requirements and translate them into technical solutions.

- Develop and maintain data quality checks and ensure data accuracy and consistency.

- Optimize data pipelines for performance, scalability, and cost-effectiveness.

- Collaborate with data analysts, data scientists, and other stakeholders to ensure data availability and accessibility.

- Implement and maintain CI/CD pipelines for data pipelines using Azure DevOps.

- Troubleshoot and resolve data-related issues.

- Stay abreast of the latest advancements in cloud data engineering and best practices.

- Mentor and guide junior data engineers.

Minimum Qualifications :

Certifications :

- Azure Data Engineering Associate Certification (required)

- Databricks Data Engineering Certification (required)

Educational Requirements :

- Bachelor's degree in Computer Science or related field.

Experience :

- 7 years of hands-on experience in designing and developing distributed data pipelines.

- 5 years of experience in Azure data services.

5 years of experience with :

- Python

- SQL

- Object-Oriented Programming (OOP)

- ETL (Extract, Transform, Load) processes

- Unit testing

Skills :

- Data integration experience with APIs, web services, and queues.

- Proficient in Azure DevOps and CI/CD pipelines.

- Familiarity with agile methodologies, JIRA, and Confluence.

Desired Skills & Experience :

- Experience with Spark, Scala, and other big data technologies.

- Experience with data warehousing and data lake architectures.

- Experience with data security and privacy best practices.

- Experience with cloud-native technologies (containers, serverless computing).

- Experience with data visualization tools (Power BI, Tableau