Azure Data Engineer

Job Summary

We are looking for highly skilled Azure Data Engineers to join our dynamic team. The candidates will be responsible for designing, implementing, and optimizing data pipelines and solutions on the Microsoft Azure platform. In addition to expertise in Azure services like Azure Data Factory (ADF), Databricks, Azure Data Lake Storage (ADLS), Fabric, and Synapse, experience working with unstructured data, real-time analytics, and Power BI exp. will be a strong advantage.

Key Responsibilities

1. Data Pipeline Development:
• Design, build, and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks.
• Optimize data pipelines for performance and scalability.
2. Data Integration and Storage:
• Integrate data from diverse sources into Azure Data Lake Storage (ADLS).
• Ensure efficient storage and retrieval of structured and unstructured data.
3. Collaboration and Support:
• Collaborate with data analysts, data scientists, and business stakeholders to understand requirements.
• Provide support and troubleshooting for data solutions.
4. Azure Services Expertise:
• Leverage Azure Fabric and other related Azure tools to enhance data engineering workflows.
• Implement monitoring and logging solutions to ensure system reliability.
5. Best Practices and Security:
• Ensure data solutions adhere to best practices in security, governance, and compliance.
• Implement data quality checks and error handling mechanisms.

Qualifications and Skills

• Role 1 (3-5 years):
• 3-5 years of hands-on experience with Microsoft Azure Data Engineering services.
• Proficiency in ADF, Databricks, ADLS, Synapse, and working with unstructured data.

• Role 2 (5-7 years):

• 5-7 years of experience in designing and delivering end-to-end data engineering solutions on Azure.
• Deep expertise in ADF, Databricks, ADLS, Synapse, real-time analytics, and unstructured data processing.

• Experience in real-time analytics and stream processing solutions.
• Knowledge of data warehousing concepts and data modelling.
• Strong SQL and Python skills for data manipulation and automation.
• Hands-on experience with Power BI is a significant advantage.
• Familiarity with CI/CD practices for data projects is a plus.

What We Offer

• Opportunity to work with cutting-edge technologies in Azure and real-time analytics.
• A collaborative and innovative work environment focused on growth and development.