Cloud 9 Infosystems - Azure Fabric Engineer - Data Factory/Databricks (5-7 yrs)

Responsibilities :

- Design, develop, and deploy end-to-end data solutions using Microsoft Azure Fabric, including its various components like Data Factory, Databricks, and Synapse Analytics within the Fabric environment.

- Create and maintain scalable and efficient data pipelines using Azure Fabric, Azure Data Factory, Azure Databricks, and Synapse Analytics for data ingestion, transformation, and loading.

- Implement and optimize data solutions based on the Medallion Architecture (Bronze, Silver, Gold layers) to ensure data quality, consistency, and efficient data processing.

- Design and implement data ingestion processes for diverse data sources (structured, semi-structured, and unstructured) into Azure Data Lake Gen2, utilizing Azure Fabric's capabilities.

- Develop and maintain data models, implement ETL/ELT processes, and design data warehousing solutions within the Azure Fabric ecosystem.

- Optimize data pipelines and queries for performance and scalability, leveraging Azure Fabric's performance tuning features.

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver effective data solutions.

- Document data pipelines, processes, and architectures, and adhere to best practices for data engineering and development within Azure Fabric.

- Identify and resolve data pipeline issues, provide ongoing support, and ensure data availability and reliability.

Required Skills and Experience :

- Proven hands-on experience in designing and implementing data solutions using Microsoft Azure Fabric.

- Deep understanding and practical experience with Azure Data Lake Gen2, Azure Data Factory, Azure Databricks, and Synapse Analytics, specifically within the Azure Fabric environment.

- Extensive experience in implementing the Medallion Architecture for building data lakes and data warehouses.

- Strong proficiency in SQL, Python, and PySpark for data processing, transformation, and analysis.

- Solid understanding of data modeling principles, ETL/ELT processes, and data warehousing concepts.

- Experience in ingesting data from various sources, including databases, APIs, and file systems.

- Excellent analytical and problem-solving skills, with the ability to troubleshoot and resolve complex data issues.

- Strong written and verbal communication skills, with the ability to effectively communicate technical concepts to both technical and non-technical audiences.

Preferred Qualifications :

- Azure certifications (e.g, Azure Data Engineer Associate).

- Experience with data governance and security best practices.

- Knowledge of DevOps practices and CI/CD pipelines