Azure Data Lead

Qualifications:

1. Bachelor's degree in Computer Science, Information Technology, or a related field.
2. Proven experience as a Data Engineer with a focus on Azure cloud technologies.
3. In-depth knowledge of Azure Data Factory, Azure SQL Database, Azure Databricks, and other relevant Azure services.
4. Strong proficiency in SQL, Python, and/or other scripting languages.
5. Experience with data modeling, schema design, and database performance tuning.
6. Familiarity with data warehousing concepts and best practices.
7. Excellent problem-solving and analytical skills.
8. Effective communication and collaboration skills.
9. Relevant certifications such as Microsoft Certified: Azure Data Engineer Associate are a plus.

Responsibilities:

1. Solution Design and Architecture:

• Collaborate with stakeholders to understand business requirements and design scalable data solutions on the Azure platform.
• Create and maintain data architecture diagrams and documentation.

1. Data Ingestion and ETL:

• Develop and implement ETL processes using Azure Data Factory, Azure Databricks, and other relevant tools.
• Design and optimize data pipelines to ensure efficient data processing and transformation.

1. Data Storage and Management:

• Implement and manage Azure SQL Database, Azure Data Lake Storage, and other Azure data storage solutions.
• Ensure data integrity, security, and compliance with industry best practices and organizational standards.

1. Data Transformation and Processing:

• Utilize Azure Databricks and other tools for data transformation and processing.
• Optimize queries and data processing workflows for performance and efficiency.

1. Monitoring and Optimization:

• Implement monitoring solutions to track the performance and health of data solutions.
• Identify and resolve performance bottlenecks and optimize data processing workflows.

1. Collaboration and Communication:

• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders.
• Communicate effectively with technical and non-technical audiences, providing insights and recommendations.

1. Automation and Scripting:

• Develop scripts and automation processes to streamline data engineering tasks.
• Implement infrastructure as code (IaC) for managing Azure resources.