Azure Data Engineer

Role & responsibilities

• Design, develop, and maintain complex data pipelines on Azure to support business requirements.
• Migrate data from on-premises and cloud-based systems to Azure platforms.
• Implement and optimize dimensional data models and data warehouses for scalable and efficient data processing.
• Work extensively with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake, and Azure SQL Database to manage and process large datasets.
• Write efficient and scalable code using Python, SQL, and optionally Scala, to support data engineering workflows.
• Develop and maintain ETL/ELT processes and leverage big data frameworks such as Apache Spark for large-scale data processing.
• Set up and manage Azure DevOps pipelines, Git repositories, and CI/CD processes for seamless deployment and automation.
• Ensure compliance with Azure security best practices, including implementing AAD, Key Vault, and RBAC for secure data access.
• Collaborate with cross-functional teams to identify, develop, and implement data solutions to meet organizational needs.
• Monitor and troubleshoot data workflows, ensuring high performance and reliability.

Preferred candidate profile

• Experience: Minimum 5 years of experience as an Azure Data Engineer.
• In-depth knowledge of Azure data services (Data Factory, Databricks, Synapse Analytics, Data Lake, SQL Database).
• Strong programming skills in Python and SQL. Familiarity with Scala is a plus.
• Expertise in data modeling and ETL/ELT processes.
• Hands-on experience with big data frameworks, such as Spark.
• Familiarity with Azure DevOps, Git, and CI/CD implementation.
• Understanding of Azure security practices, including Azure Active Directory (AAD), Key Vault, and Role-Based Access Control (RBAC).
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.
• Ability to adapt to changing business needs and work effectively in an offshore setup.