Azure Data Engineer - ETL/Data Pipeline (2-6 yrs)

Hiring for Data Engineer role with Randstad.

Skills : Python, Pyspark, SQL, Azure

Experience : 2-6 Years

Location : Gurgaon (WFO)

Need only Immediate Joiners.

Job Description :

As a Data Engineer at [Company Name], you will work closely with data scientists, analysts, and business teams to build robust, scalable, and efficient data pipelines. You will be responsible for handling ETL processes, designing and developing API integrations, and leveraging Azure cloud services to manage large data sets. The ideal candidate will have strong expertise in Python, PySpark, SQL, and a deep understanding of various Azure services (Azure Functions, Azure Data Factory, Azure API Services, Azure DevOps).

Key Responsibilities :

- ETL Development & Maintenance : Design, build, and maintain efficient, automated ETL pipelines to extract, transform, and load data from multiple sources using Azure Data Factory, PySpark, and other tools.

- Cloud Platform Utilization : Work with Azure services, including Azure Functions, Azure Data Factory, Azure API Services, and

- Azure DevOps, to develop cloud-based solutions and automate data processing workflows.

- Data Integration : Implement and optimize data integration solutions to pull data from various sources (APIs, databases, etc.) using polling mechanisms and manage data ingestion.

- API Development & Middleware : Develop, deploy, and maintain APIs for data integration and ensure data flows seamlessly between systems. Implement API services using Azure API Management.

Required Skills & Qualifications :

- Programming Languages : Strong proficiency in Python and PySpark for data processing, scripting, and automation.

- SQL Expertise : Deep knowledge of SQL for data querying, transformation, and integration tasks.

- Azure Expertise : Experience working with various Azure services, including : Azure Data Factory (ADF) for orchestrating ETL pipelines, Azure Functions for serverless data processing, Azure API Services for building and managing APIs, Azure DevOps for version control, CI/CD, and automated deployments.

- ETL and Data Pipelines : Solid understanding of ETL processes and experience with data extraction, transformation, and loading (ETL) using cloud tools.

Education & Experience :

- Bachelor's degree in Computer Science, Engineering, or related field.

- 3+ years of hands-on experience in data engineering, cloud services, and ETL development.