Senior Azure Data Engineer - SQL/ETL (7-9 yrs)

Job Description :

We are seeking a highly skilled Senior Azure Data Engineer with expertise in Azure Data Factory (ADF), Azure Databricks, PySpark, SQL, ADO, Power BI (Basics), and Unity Catalog. The ideal candidate will have strong experience in building scalable data pipelines, optimizing data lakes, and implementing data governance solutions using Azure technologies.

Key Responsibilities :

1. Data Engineering & Pipeline Development :

- Design and develop ETL/ELT pipelines using Azure Data Factory (ADF) & Databricks.

- Implement PySpark-based transformations to process large-scale datasets efficiently.

- Optimize data ingestion, transformation, and storage in Azure environments.

2. Data Modeling & SQL Expertise :

- Design and implement data models for analytics and reporting use cases.

- Write and optimize complex SQL queries for data warehousing and data processing.

- Implement best practices in SQL performance tuning and indexing strategies.

3. Azure Data Services & Governance :

- Manage Azure Data Lake Storage (ADLS) and Delta Lake architectures.

- Implement Unity Catalog for data governance, security, and access control.

- Ensure compliance with data privacy, governance, and security best practices.

4. DevOps & CI/CD Implementation :

- Utilize Azure DevOps (ADO) for version control, CI/CD, and release management.

- Implement automated deployment of data pipelines using DevOps best practices.

5. Reporting & Analytics Support :

- Provide basic support in Power BI for data visualization and reporting.

- Work with data analysts to ensure seamless data integration with Power BI dashboards.

Must-Have Skills & Qualifications :

- 7 to 9 years of experience in Azure Data Engineering & Cloud-based ETL/ELT processes.

- Strong expertise in Azure Data Factory (ADF), Databricks, PySpark, and SQL.

- Hands-on experience with Azure Data Lake, Delta Lake, and Unity Catalog.

- Proficiency in ADO (Azure DevOps) for CI/CD and pipeline automation.

- Basic knowledge of Power BI for data visualization and analytics.

- Strong experience in optimizing data pipelines for performance and cost-efficiency.

- Excellent communication skills and ability to work in flexible timings.

Preferred Qualifications :

- Experience with Delta Live Tables & Structured Streaming in Databricks.

- Knowledge of Azure Synapse Analytics for data warehousing.

- Understanding of MLflow & MLOps for AI/ML model deployment (nice to have).

- Azure Certifications (e.g., DP-203: Azure Data Engineer) are a plus.

Benefits :

- Opportunity to work with cutting-edge Azure & Databricks technologies.

- Flexible work timings and remote/hybrid options available.

- Work on enterprise-level projects with global exposure.