Azure Data Engineer

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure storage solutions such as Blob Storage, Cosmos DB, etc.
  
• Collaborate with cross-functional teams to understand business requirements and design scalable data architectures that meet those needs.
  
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Microsoft SQL Server databases.
  
• Implement big data processing workflows using PySpark on Azure Databricks for real-time analytics applications.
  
• Ensure high availability of critical systems by implementing monitoring tools like Prometheus/Grafana/New Relic.
  

Desired Candidate Profile

• 5-10 years of experience in designing and developing large-scale data warehousing solutions on Azure platform.
  
• Strong expertise in working with ADF, SQL, Python/PySpark programming languages.
  
• Experience with cloud-based technologies such as Azure Databricks, Blob Storage, Cosmos DB etc.