Expert Data Engineer

As part of the central Data Engineering unit, you will be responsible for managing and coordinating external and distributed engineers, while providing community support, training, and governance to drive best practices across the engineering community within BC.
Key Responsibilities Include.
Design and build robust and scalable data pipelines to load and transform data from various source systems.
Collaborate with data scientists, analysts, and business stakeholders to understand their data requirements and build the right solutions.
Implement validation processes to ensure data is accurate, consistent, and accessible.
Build automation for repetitive tasks related to data analytics.
Monitor and improve pipeline performance, identifying and resolving bottlenecks.
Perform root cause analysis and implement corrective measures.
Document data engineering processes, systems, and procedures.
Identify opportunities to optimize pipelines for performance and cost efficiency.
Manage and coordinate BCs external and distributed engineers, ensuring effective collaboration and integration.
Provide community support, training, and governance to drive best practices across the data engineering community.
Lead the development and implementation of global frameworks and patterns for data ingestion and pipelines, working closely with all other internal and external Data Engineer roles.
About You.
Advanced knowledge of data engineering processes, with a minimum of 7 years experience (senior).
Comprehensive understanding of data engineering patterns and best practices for pipeline orchestration.
Extensive experience in developing data analytics solutions on Azure.
Strong background in designing and building efficient, reliable, and automated data pipelines, ETL workflows, data warehousing, and Big Data processing, with experience in respective technologies like Airflow, dbt, etc Proficiency in Spark and Databricks Lakehouse technologies.
Hands-on expertise with Python or PySpark.
In-depth knowledge of Azure Stack components like Azure Databricks, Azure Data Factory, Azure Synapse, and ADLS Gen2 from a data engineering perspective.
Familiarity with utilizing DevOps and DataOps methodologies to enhance development and deployment practices (e g, CI/CD).
Experience in managing and coordinating data engineers, providing community support, training, and governance to drive best practices.
Expertise in leading the development and implementation of global frameworks and patterns.
Excellent problem-solving skills and a structured way of working.
Collaborates well across diverse and globally distributed teams, with the ability to build and maintain positive relationships across different levels and functions of the organization.