Azure Data Factory Developer (6-10 yrs)

Total Yrs of Experience : 6 to 10 yrs.

Job Summary :

We are seeking an experienced MS Azure ADF Developer with a strong background in API design, data mapping, and ETL processes. The ideal candidate will have expertise in handling large data sets and optimizing data syncs. The role involves designing and developing APIs to integrate platforms, re-testing platforms using Azure ADF, and improving the performance of existing data mapping processes. The candidate should be proficient in managing large datasets and ensuring quick data synchronization.

Key Responsibilities :

- Design and develop APIs to transfer data between platforms.

- Re-test and optimize platforms using Azure ADF for better performance.

- Revise and enhance the current data mapping document to increase processing speed and reduce data sync time.

- Manage large datasets (1.5 million products) and implement efficient strategies to sync only the changes from previous drops, with a data sync target of under 15 minutes.

- Write and optimize ETL processes, leveraging best practices for efficient data movement.

- Collaborate with cross-functional teams to ensure data integration and platform performance are up to standards.

Required Skills :

- 6+ years of experience in API design and development.

- Extensive experience with Azure Data Factory (ADF).

- Strong knowledge of ETL technologies and best practices.

- Experience handling large datasets and data synchronization strategies.

- Proficiency in data mapping and re-testing using Azure ADF.

- Expertise working with Databricks Workspace Interface, Notebook, python and deep SQL exp.

- The candidate should have experience in developing scalable data architecture, data transformation process and analytical solution supporting business needs for data management.

- Experienced working on power BI report Development projects (SSE) and having extensive data engineering background using Azure Databricks expertise in Azure Data Bricks and Notebook experienced in Azure Data factory

- Python Azure Cluster Architecture. T-SQL Programming Git, TFS, VST

- Similarity with DevOps practices, including CI/CD pipelines using tools such as Azure DevOps and GIT

Good to have Skills : Azure Synapse/Azure Fabric Power BI