Azure Data Engineer

Position Summary

Someone who is responsible for the Azure BigData platform in collaboration with vendor partner and BigData development team. Someone with hands on working experience in Azure Data Analytics using ADLS, Delta Lakes, Cosmos DB, Python and Spark/Scala programing to facilitate a strong, robust and flexible platform.

Job Responsibilities

• Assist BigData development & Data science team by writing basic Python scripts, Spark & PySpark etc.
• Assist BigData development and Data science team in Data Ingestion projects in Azure environment
• Configure and manage I-a-a-S, P-a-s-S
• Experience of Azure cloud components:
• Azure Data Factory
• Azure Data Lake
• Azure Data Catalogue
• Azure LogicApps & FunctionApps
• Azure Synapse Analytics
• Azure Databricks
• Azure EventHub
• Azure Functions
• Azure SQL DB

Knowledge, Skills and Abilities

Education

• Bachelor's degree in Computer Science, Engineering, or related discipline

Experience

• 5 to 8 years of solutions design & development experience
• Experience in building Data Ingestion/Transformation pipelines on Azure Cloud
• Experience in BigData Tools like Spark, Delta Lakes, ADLS, Azure Synapse /Databricks
• Proficient understanding of distributed computing principles

Knowledge and skills (general and technical)

• Spark, Scala, Python
• Azure Cosmos DB, Mongo DB
• Azure Data Factory
• ADLS-Gen2
• Azure Data Lake
• Azure Data Catalogue
• Azure LogicApps & FunctionApps
• Azure Synapse Analytics
• Azure Databricks
• Azure EventHub
• Azure Functions
• Azure SQL DB