Azure Data Engineer (5-7 yrs)

We are seeking a skilled Azure Data Engineer with 5-7 years of experience in designing, developing, and maintaining data solutions on Microsoft Azure. The ideal candidate will have expertise in Azure Data Factory (ADF), Azure Databricks (ADB), and Synapse Analytics. The role involves working with large-scale data processing, ETL pipelines, cloud connectivity, and database management to support business intelligence and analytics.

Key Responsibilities :

- Develop and maintain ETL pipelines using Azure Data Factory (ADF), Azure Databricks (ADB), and Synapse Analytics.

- Establish and optimize cloud connectivity between Azure Data Lake Storage (ADLS), ADF, Synapse, and Databricks.

- Design and implement data extraction, transformation, and loading (ETL) processes for structured and unstructured data.

- Work with big data processing frameworks (Spark with PySpark or Scala) to manage large-scale data transformations.

- Design and manage large SQL data marts, ensuring efficient data retrieval and reporting.

- Develop T-SQL and PL/SQL scripts to support data processing and integration.

- Monitor and troubleshoot database performance, security, and storage issues.

- Implement backup, archiving, and capacity planning strategies for database management.

- Develop Python scripts to automate data workflows and processing.

- Utilize PySpark or Scala for big data transformations in Azure Databricks.

- Implement Azure Data Factory framework for orchestrating data pipelines.

- Ensure data security and compliance with industry best practices.

- Manage user access control, authentication, and permissions in Azure data platforms.

- Identify and mitigate potential security risks in cloud-based data solutions.

- Work closely with data analysts, BI developers, and business stakeholders to understand data requirements.

- Provide technical support and troubleshooting for data-related issues.

- Optimize existing data workflows for better efficiency and scalability.

- Document technical processes, solutions, and best practices.

Required Skills & Expertise :

- Expertise in Azure Data Factory (ADF), Azure Databricks (ADB), Synapse Analytics.

- Strong experience in Python, PySpark or Scala, and T-SQL.

- Expertise in PL/SQL, large-scale SQL data marts, and database security.

- Experience in data extraction, transformation, and loading (ETL) processes.

- Hands-on experience establishing Azure cloud integrations (ADLS, ADF, Synapse, Databricks).

- Knowledge of performance tuning, capacity planning, and troubleshooting large databases.

- Understanding of data security best practices and user access management.

- Experience in financial institution data marts.

- Strong analytical and problem-solving abilities.

- Proactive, organized, and detail-oriented.

- Ability to work independently and in a team environment.

- Excellent verbal and written communication skills.

- Work on cutting-edge Azure data solutions.

- Opportunity to handle large-scale data processing and analytics.

- Collaborative and growth-oriented work environment.

- Competitive compensation and career advancement opportunities.