Azure Lead

Lead Data Engineer utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle the distribution of data types and volumes in support of data architecture design.

A Senior Data Engineer designs and oversees the entire data infrastructure, data products, and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost-effective.

Key Responsibilities:

• Design and oversee the entire data architecture strategy.
• Mentor junior data architects to ensure skill development in alignment with the team strategy.
• Design and implement complex scalable, high-performance data architectures that meet business requirements.
• Model data for optimal reuse, interoperability, security, and accessibility.
• Develop and maintain data flow diagrams and data dictionaries.
• Collaborate with stakeholders to understand data needs and translate them into technical solutions.
• Ensure data accessibility through a performant, cost-effective consumption layer that supports use by citizen developers, data scientists, AI, and application integration.
• Ensure data quality, integrity, and security across all data systems.

Qualification:

• Experience in Erwin, Azure Synapse, Azure Databricks, Azure DevOps, SQL, Power BI, Spark, Python, and R.
• Ability to drive business results by building optimal cost data landscapes.
• Familiarity with Azure AI/ML Services, Azure Analytics: Event Hub, Azure Stream Analytics, Scripting: Ansible
• Experience with machine learning and advanced analytics.
• Familiarity with containerization and orchestration tools (e. g. , Docker, Kubernetes).
• Understanding of CI/CD pipelines and automated testing frameworks.
• Certifications such as AWS Certified Solutions Architect, IBM certified data architect, or similar are a plus.

Azure Synapse, Adf, Adb, Python, Pyspark - Azure, Azure Databricks, Azure Dat Factory, Azure, Adf And Adb