Azure Data Engineer - Cloud Infrastructure

- Design, build, and implement cloud infrastructure and services using Azure.

- Implement and optimize CI/CD pipelines using tools like GitHub, Jenkins, Azure DevOps etc.

- Experience with exposing data to end-users via Power BI, Azure API Apps.

- Knowledge of Azure Data & Analytics PaaS Services: Azure Data Factory, Azure Data Lake, Azure Synapse Analytics

- Designing and implementing data storage solutions on Azure

- Building and maintaining data pipelines for data integration and processing

- Developing and maintaining data models and schemas

- Experience in turning business use cases and requirements into technical solutions.

- Experience in business processing mapping of data and analytics solutions

- Experience in managing teams, being a technical coach.

- Experience with agile delivery methodologies, proof of concepts and prototyping

- Ability to conduct data profiling, cataloguing, and mapping for technical design and construction of technical data flows

- Designing and implementing scalable and secure data processing pipelines using Azure Data Factory, Azure Databricks, and other Azure services.

- Managing and optimizing data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB.

- Developing data models and maintaining data architecture to support data analytics and business intelligence reporting.

- Ensuring data quality and consistency through data cleaning, transformation, and integration

processes.

- Monitoring and troubleshooting data-related issues within the Azure environment to maintain high availability and performance.

- Collaborating with data scientists, business analysts, and other stakeholders to understand data requirements and implement appropriate data solutions.

- Implementing data security measures, including encryption, access controls, and auditing, to protect sensitive information.

- Automating data pipelines and workflows to streamline data ingestion, processing, and distribution tasks.

- Utilizing Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making.

- Keeping abreast of the latest Azure features and technologies to enhance data engineering processes and capabilities.

- Documenting data procedures, systems, and architectures to maintain clarity and ensure compliance with regulatory standards.

- Providing guidance and support for data governance, including metadata management, data lineage, and data cataloging.