Azure Data Engineer - Python/ETL (8-12 yrs)

Job Title : Azure Data Engineer

Location : Hyderabad (Hybrid)

Experience : 8+ years

Job Description :

We are seeking a highly skilled Azure Data Engineer with expertise in ADF (Azure Data Factory), SQL, ABD (Azure Databricks - UC + Delta Tables), PySpark, Python, and Synapse Analytics. The ideal candidate will be responsible for designing, implementing, and optimizing data solutions on Azure to support business intelligence, analytics, and data-driven decision-making.

Key Responsibilities :

- Develop and maintain data pipelines using Azure Data Factory (ADF) to ingest, transform, and store data efficiently.

- Work extensively with SQL to design and optimize data models, write complex queries, and manage data storage solutions.

- Implement and optimize data processing workflows using Azure Databricks (ABD) with Unity Catalog (UC) and Delta Tables (DT).

- Develop data processing solutions using PySpark and Python to handle large-scale data processing.

- Work with Azure Synapse Analytics to design and manage large-scale data warehouse solutions.

- Ensure data quality, integrity, and governance through best practices and compliance with enterprise standards.

- Optimize performance of data pipelines, ETL processes, and storage solutions to enhance efficiency.

- Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements.

- Monitor and troubleshoot data pipeline issues and ensure smooth data processing operations.

Required Skills & Experience :

- 8+ years of experience in data engineering with expertise in Azure cloud services.

- Strong hands-on experience in Azure Data Factory (ADF) and SQL (very strong skills required).

- In-depth knowledge of Azure Databricks (Unity Catalog + Delta Tables) and data processing using PySpark & Python.

- Experience with Azure Synapse Analytics for large-scale data warehousing.

- Strong understanding of data modeling, ETL processes, and data governance principles.

- Expertise in optimizing data workflows and performance tuning for large datasets.

- Knowledge of Azure security and compliance standards related to data engineering.

- Experience working in Agile and DevOps environments with CI/CD for data pipelines.

Preferred Qualifications :

- Experience with Azure Data Lake Storage (ADLS) and integration with various Azure services.

- Knowledge of Power BI or other visualization tools is a plus.

- Certification in Microsoft Azure Data Engineering (DP-203) is highly desirable.