Saxon - Azure Data Engineer - Databricks/Data Factory (3-5 yrs)

About the Role :

We are seeking a highly skilled and experienced Data Engineer with a strong focus on Azure cloud technologies to join our growing team. In this critical role, you will be responsible for designing, building, and maintaining data pipelines and solutions on the Azure platform. You will work closely with data scientists, analysts, and other engineers to deliver high-quality data solutions that drive business insights and decision-making.

Key Responsibilities :

Azure Data Engineering :

- Design, develop, and maintain data pipelines and ETL processes using Azure Synapse Analytics, Azure Data Factory, and Azure Databricks.

- Provision and configure Azure cloud resources for data storage and processing.

- Implement data quality checks and monitoring to ensure data accuracy and reliability.

- Optimize data pipelines for performance and scalability.

- Build and deploy data science and machine learning pipelines.

Database Management :

- Work with Azure SQL Server, PostgreSQL, or similar RDBMS platforms.

- Understand database structure principles and performance troubleshooting.

- Design and implement efficient database schemas.

Big Data Processing :

- Develop and implement distributed data processing solutions for big data batch and streaming pipelines.

- Utilize Azure Databricks for large-scale data processing and analysis.

Data Visualization :

- Familiarity with data visualization tools like Power BI. Develop and maintain data visualizations and dashboards.

CI/CD :

- Configure and maintain CI/CD pipelines for Azure Data Factory, Azure Databricks, and Azure Synapse Analytics using Azure DevOps.

Collaboration :

- Collaborate effectively with data scientists, analysts, and other engineers.

- Communicate technical concepts clearly and effectively.

Required Skills & Experience :

- Hands-on experience as a Data Engineer or similar role.

- Proven experience with Azure cloud services, including : Azure Synapse Analytics Azure Data Factory Azure Databricks

- In-depth understanding of database structure principles and performance troubleshooting.

- Experience with Azure SQL Server, PostgreSQL, or similar RDBMS platforms.

- Experience in distributed data processing for big data batch and streaming pipelines.

- Experience in constructing data science and machine learning research and production pipelines.

- Familiarity with data visualization tools like Power BI.

- Experience in configuring CI/CD pipelines for Azure Data Factory, Azure Databricks, and Azure Synapse Analytics using Azure DevOps.

Preferred Skills :

- Experience with other Azure data services (e.g., Azure Data Lake Storage, Azure Cosmos DB).

- Knowledge of data governance and security best practices.

- Experience with Agile development methodologies.

- Experience with scripting languages (e.g., Python, SQL).