Azure Data Engineer - ETL/SQL (3-5 yrs)

Role : Senior Data Engineer

Job Description :

We are looking for a highly skilled Senior Data Engineer with expertise in Azure Synapse Analytics, Azure Databricks, and a strong understanding of cloud data platforms.

The ideal candidate will have hands-on experience in building and optimizing data pipelines, automating data integration processes, and working with large-scale data lakes and SQL systems.

Key Responsibilities :

- Design, develop, and maintain scalable data pipelines in Azure Synapse Analytics and Azure Databricks.

- Implement and optimize data integration processes, ETL automation, and performance tuning.

- Work extensively with Azure Cloud, Azure DataLake, and Azure SQL to manage and store large data sets.

- Utilize Elastic Search for data querying and indexing.

- Collaborate with business intelligence teams to deliver insights using tools like MicroStrategy or other BI platforms.

- Develop solutions using Python and PySpark to support data transformation and integration tasks.

- Ensure system performance and data quality through regular testing, monitoring, and maintenance.

- Collaborate with cross-functional teams, including AI/ML experts, to leverage advanced analytics and machine learning techniques (good to have).

Requirements :

- Proficiency in Azure Synapse Analytics and Azure Databricks.

- Strong experience with Azure Cloud, Azure DataLake, and Azure SQL.

- Expertise in ETL tools, data pipeline automation, and performance optimization.

- Hands-on experience with Elastic Search.

- Experience with BI tools such as MicroStrategy (or similar).

- Strong Python and PySpark development skills (good to have).

- Knowledge of AI/ML fundamentals (good to have).

- Strong problem-solving and collaboration skills.