Azure Data Engineer - SQL/PySpark (3-6 yrs)

Job Overview :

We are looking for a skilled Azure Data Engineer to design, develop, and optimize data pipelines and solutions within the Azure ecosystem.

The ideal candidate will possess expertise in Azure Databricks, Azure Data Factory, Azure Cloud services, and have strong proficiency in Data Modelling, SQL, and PySpark.

Key Responsibilities :

- Design and implement scalable data pipelines and ETL/ELT processes using Azure Data Factory and Databricks.

- Develop and optimize data models to support reporting and analytics needs.

- Collaborate with cross-functional teams to understand data requirements and deliver robust data solutions.

- Write efficient and maintainable code in PySpark and SQL for data transformations and processing.

- Ensure data quality and integrity through validation and testing.

- Monitor, troubleshoot, and enhance the performance of Azure data solutions.

- Leverage Azure Cloud services to implement secure, reliable, and high-performance data systems.

- Maintain technical documentation for all data engineering processes and solutions.

Required Skills and Qualifications :

- Proven experience as a Data Engineer working with the Azure ecosystem.

- Strong knowledge of Azure Databricks, Azure Data Factory, and Azure Cloud Services.

- Hands-on expertise in Data Modelling, including dimensional and relational designs.

- Proficient in SQL for complex data queries and transformations.

- Solid programming skills in PySpark for big data processing.

- Familiarity with CI/CD pipelines and DevOps practices for data engineering.

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration abilities.