KGiSL - Azure Data Engineer - ETL/Spark/Hadoop (9-12 yrs)

Job Description :

- Design, develop, and maintain data pipelines and solutions on the Azure platform, including data ingestion, transformation, and loading (ETL/ELT).

- Develop and implement data models and schemas for data warehousing and data lakes.

- Build and optimize data pipelines using Azure Data Factory, Azure Databricks, Azure Functions, and other relevant Azure services.

- Develop and maintain data quality checks and monitoring processes.

- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.

- Troubleshoot and resolve data-related issues.

- Stay abreast of the latest advancements in Azure data technologies and best practices.

- Contribute to the development and improvement of data engineering processes and methodologies.

- Adhere to industry best practices for data security, privacy, and governance.

Required Skills & Experience :

- 9+ years of experience in data engineering, data warehousing, and data integration.

- Strong proficiency in Azure data services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL Database, and Azure Blob Storage.

- Experience with data modeling techniques (dimensional modeling, star schema).

- Expertise in ETL/ELT processes, including data extraction, transformation, and loading techniques.

- Proficiency in scripting languages like Python and SQL.

- Experience with data quality and data profiling tools.

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration skills.

- Ability to work independently and as part of a team.

Preferred Skills :

- Experience with cloud-native technologies and microservices architecture.

- Knowledge of data governance and data security best practices.

- Experience with Agile development methodologies.

- Experience with big data technologies (Hadoop, Spark).

- Certifications in relevant Azure data technologies.