Azure Data Engineer

Data Engineer

Job Decsription

One Data Eng with good knowledge & understanding of ADF, Azure Databricks, MS SQL server, T-SQL, PySpark, and Data warehousing solutions.

Requirements

• Minimum of 5 years of experience in designing and implementing data solutions using Microsoft Azure.
  
• Understanding of cloud computing principles and architecture
  
• Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into Azure data platforms.
  
• Proficiency in Azure data services and tools, such as Azure Data Lake Storage, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL Data Warehouse
  
• Strong programming skills in languages like SQL, Python/PySpark, or PowerShell
  
  • Experience with data modeling, data warehousing, and dimensional modeling
  
• Exposure to any Source Control Management, like TFS/GIT/SVN
  
• Familiarity with Azure security features and cloud migration
  
• Knowledge of data governance, data security, and compliance best practices
  
• Experience in retail domain is added advantage.
  

Responsibilities:

• Lead workshops with many teams to define data ingestion, validation, mining, engineering, modeling, visualization, AI, and analytics
  
• Participate in design sessions, demos and prototype sessions, testing and training workshops with business users and other IT associates
  
• Liaise with offshore team and clients for resolving technical dependencies, issues, and risks
  
• Create ADF & Databricks data transformation pipelines from scratch
  
• Implement data security at Storage and data movement layers
  
• Assure that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements
  * The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)
  * Load transformed data into storage and reporting structures in destinations including data warehouse