Azure Data Architect - Data Factory/Databricks (10-15 yrs)

Role Overview :

We are seeking a highly skilled and experienced Azure Data Architect with 10+ years of hands-on experience in building, managing, and optimizing large-scale data solutions. In this role, you will design, implement, and maintain data architectures on Microsoft Azure, ensuring seamless data pipelines and analytics capabilities. As a senior member of our data engineering team, you will collaborate closely with data scientists, analysts, and other engineering teams to drive impactful business insights and innovation along with leading a team of junior data engineers & guiding them to achieve the project deliverables

Key Responsibilities :

- Collaboration : Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data needs and translate them into effective data engineering solutions.

- Data Pipeline Design : Design and implement robust data pipelines that support SRM initiatives, focusing on data ingestion, transformation, and storage for enterprise-scale applications.

- Architecture Development : Lead the architecture and design of data solutions, ensuring scalability, performance, and reliability to meet business demands.

- Technology Selection : Evaluate and select appropriate data engineering tools and technologies to optimize data processing workflows and enhance overall efficiency.

- Data Integration : Develop and maintain data integration processes to ensure seamless flow of information across various systems, including ERP, CRM, and other business applications.

- Cloud Solutions : Design and implement cloud-based data architectures using platforms such as AWS, Azure, or Google Cloud to facilitate data storage, processing, and analytics.

- Data Quality Assurance : Establish data quality standards and practices to ensure the integrity and accuracy of data throughout the pipeline.

- Performance Optimization : Monitor and optimize data workflows for performance, making necessary adjustments to improve speed and efficiency.

- Documentation and Best Practices : Create and maintain comprehensive documentation of data architectures, processes, and best practices for the engineering team.

- Mentorship : Provide guidance and mentorship to junior data engineers, fostering a culture of continuous learning and improvement within the team.

Required Skills and Qualifications :

- 10+ years of experience in data engineering, with at least 6+ years of experience working with Azure-based architecture solutions in delivering mid to high scale projects

- Strong hands-on expertise in Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake, Azure SQL Database

- Proven experience in designing and implementing end-to-end data architecture, including ingestion, transformation, and storage in the cloud.

- In-depth knowledge of SQL, data warehousing, and ETL processes.

- Experience with data security, data privacy, and governance standards within the Azure cloud platform.

- Proficiency in programming languages like Python/ Pyspark

- Good understanding of Azure DevOps, version control, and CI/CD pipelines for automation.

- Experience with any of the big data technologies such as Hadoop, Spark, and their integration into Azure-based environments.

- Ability to work with diverse data types and sources, including relational, NoSQL, and flat-file data systems.

- Strong experience of cloud architecture principles and best practices for scalability and high availability in Azure.

- Excellent communication skills and the ability to collaborate effectively with internal teams and clients.

Preferred Skills :

- Experience with machine learning models and deployment in Azure (Azure Machine Learning).

- Knowledge of Power BI for data visualization and reporting.

- Understanding of containerization and orchestration technologies like Docker and Kubernetes.

- Certifications in Azure Data Engineering (Azure DP-203) or relevant Azure certifications.

Educational Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, Data Science, or a related field.