Senior Data Engineer - Azure SQL (5-8 yrs)

Senior Data Engineer and we'll allow you to grow and develop your capabilities as well as support you through Microsoft certifications.

Responsibilities :

- Developing low to high-complexity, secure, governed, high-quality, and efficient data pipelines from a variety of on and off-premise, internal, and external data sources.

- Ensuring that data is cleansed, mapped, transformed, and optimized for storage to meet requirements for business and technical use cases.

- Building data observability and data quality by design into all Data pipelines, promoting self-testing pipelines that proactively

identify processing issues or discrepancies.

- Take all requirements into account when creating a proposal.

- Understanding and evaluating the organizational impact of new technologies and digital services.

- Troubleshooting and resolving issues.

- You may be building solutions that transform data into data lake storage areas, physical database models, and reporting structures across data lakes, data warehouses, business intelligence systems, and analytics applications.

- You may also be required to carry out unit testing of your own code and peer testing of others to ensure appropriate quality.

Requirements :

- Experience working with large, event-based, datasets in an enterprise environment.

- Minimum 4 years within an Azure or SQL Data Engineering environment

- Engineering and database skills in SQL Server and/or Azure (Data Factory, Synapse) and the associated ETL technologies.

- Experience developing, driving, and maintaining Data Integration Roadmaps and best practices.

- Overall responsibility for all technical business requirements gathering deliverables and implementation.

- Ability to deliver high to low-level designs for Data Solutions and have a good understanding of the full data lifecycle.

- Hands-on experience in designing and developing scripts for custom ETL processes and automation in Azure Data Factory, Azure Databricks, Azure Synapse, Python, Pyspark, etc.

- Hands-on experience designing and developing stored procedures/functions and leveraging ETL packages/workflows.

- Expertise in Cloud-related Big Data integration and infrastructure tech stack using Azure Databricks and Apache Spark framework.

- Well-versed with scripting languages such as Python

- Experience of Continuous Integration / Continuous Delivery (CI/CD).

- Experience working in an agile environment and self-managing teams.

- Understanding and practical knowledge of testing techniques and tools

- Desired technologies : Big Data, Azure, Spark, Python, Azure Databricks, Azure Synapse, Azure Data Factory, and SQL Server.