Senior Data Engineer - Azure Data Factory (4-8 yrs)

Job Title : Senior Data Engineer

Location : Gurgaon, India

Experience : 4 to 8 Years

Work Mode : Hybrid (3 days Work From Office)

Key Responsibilities :

- Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF) to ingest, transform, and load data into data warehouses and data lakes.

- Optimize data pipelines for performance and scalability.

- Implement data quality checks and monitoring mechanisms.

- Leverage Azure Data Bricks for data processing and analysis.

- Utilize Azure Synapse Analytics for data warehousing and business intelligence.

- Effectively use Azure Data Lake Storage (ADL) for data storage and retrieval.

- Design and implement efficient ETL processes to extract, transform, and load data from various sources (e.g, databases, APIs, files).

- Optimize ETL processes for performance and accuracy.

- Write clean, efficient, and well-documented Python code to automate tasks, process data, and build data pipelines.

- Utilize Python libraries like Pandas, NumPy, and Scikit-learn for data analysis and machine learning.

- Identify and resolve data quality issues and performance bottlenecks.

- Continuously improve data pipelines and processes to enhance efficiency and accuracy.

- Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements and deliver data-driven insights.

- Work effectively with cross-functional teams to achieve project goals.

Required Skills and Experience :

- 4-8 years of experience in data engineering or a related field.

- Proficiency in Azure Data Factory (ADF), Azure Data Bricks, Azure Synapse Analytics, and Azure Data Lake Storage (ADL).

- Strong Python programming skills, including knowledge of data manipulation and analysis libraries.

- Experience with SQL and data modeling techniques.

- Knowledge of data warehousing and data lake concepts.

- Understanding of cloud computing principles and Azure services.

- Strong problem-solving and analytical skills.

- Excellent communication and teamwork skills.

Preferred Skills :

- Experience with data visualization tools (Power BI, Tableau).

- Knowledge of machine learning and AI techniques.

- Experience with data security and privacy best practices