Lead - Azure Data Engineer

Job Summary:

We are seeking a highly skilled Lead Azure Data Engineer to design, develop, and optimize data solutions on the Azure platform. The ideal candidate will have extensive experience in Azure cloud services, data engineering, and big data technologies. This role involves leading a team, ensuring best practices, and driving data-driven solutions for business needs.

Key Responsibilities:

• Design and implement scalable and efficient data pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.
• Develop and manage ETL/ELT workflows for data ingestion, transformation, and storage.
• Optimize and maintain data lakes, data warehouses, and databases in Azure.
• Implement data governance, security, and compliance best practices.
• Work with stakeholders to understand business needs and translate them into technical solutions.
• Lead and mentor a team of data engineers, ensuring adherence to best practices.
• Automate data workflows using Azure Functions, Logic Apps, or Pyspark.
• Monitor and troubleshoot data pipeline performance issues.
• Collaborate with Data Scientists, BI Analysts, and Software Engineers to integrate data solutions.

Required Skills & Qualifications:

• 8+ years of experience in data engineering with at least 3+ years in Azure.
• Strong expertise in Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, and Azure SQL Database.
• Proficiency in Pyspark, SQL.
• Experience with Delta Lake, Parquet, and other modern data storage formats.
• Hands-on experience with CI/CD pipelines using Azure DevOps.
• Understanding of data modeling, warehouse architecture, and big data processing.
• Knowledge of Azure security best practices including RBAC, encryption, and data masking.
• Experience working with structured and unstructured data sources.
• Excellent communication and leadership skills.