Data Architect

Role description: We are seeking a talented and experienced Data Architect/Modeler with a strong background in Azure Cloud, Databricks, and Data Lake environments. The ideal candidate will have 5-8 years of hands-on experience designing, building, and maintaining robust data architectures, data pipelines, and models, leveraging cloud-based technologies. A deep understanding of SQL, Python, and PySpark is required to manage and optimize large-scale data systems. The candidate will play a key role in the organizations data strategy by ensuring that the data platform is scalable, efficient, and aligned with business objectives.

Key responsibilities:

Data Architecture & Modelling: Design and implement scalable, high-performance data models and architecture for both operational and analytical systems, using best practices in data management.

Azure Cloud & Databricks: Develop and optimize data solutions using Azure services (e.g., Azure Data Lake, Azure Synapse, Azure SQL Database, Azure Data Factory) and Databricks.

Data Lake Management: Oversee the structure and integration of Data Lakes, ensuring they are well-organized, optimized for performance, and support both structured and unstructured data.

Data Pipeline Development: Build end-to-end ETL/ELT pipelines using PySpark, SQL, and Databricks to transform raw data into actionable insights.

Collaboration: Work closely with data engineers, data scientists, and business stakeholders to understand data requirements and deliver solutions that meet organizational needs.

Performance Tuning & Optimization: Monitor, troubleshoot, and optimize data models, queries, and pipelines to ensure high performance and scalability.

Best Practices: Promote best practices for data management, governance, and security, ensuring the integrity and privacy of data across the platform.

Documentation & Reporting: Provide thorough documentation of data models, architecture, and pipeline designs. Offer technical support and guidance to team members as needed.

Continuous Improvement: Stay updated with emerging trends in cloud technologies, data architecture, and modelling techniques to continuously enhance data management capabilities.

Key stakeholders:

• Data Analytics team
• Data Scientist/ ML Engineer
• Product and Implementation partners
• Product Managers and delivery leads from IT horizontal teams within SAP etc.

Required Skills & Experience:

Experience: 5-8 years of experience in data architecture, modelling, or related roles.

Cloud Experience: Hands-on experience with Azure Cloud, including services like Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, and Azure Data Factory.

Databricks: Strong experience with Databricks for building and optimizing data pipelines.

Data Modelling: Expertise in designing logical and physical data models, as well as dimensional and relational models.

Programming Languages: Advanced proficiency in SQL, Python, and PySpark for data manipulation, transformation, and pipeline automation.

ETL Development: Experience in developing and optimizing ETL/ELT processes.

Data Lake Management: Familiarity with data lake best practices, including data ingestion, storage, and querying for structured and unstructured data.

Performance Optimization: Proven ability to optimize data models, queries, and pipelines for performance, scalability, and reliability.

Problem Solving: Strong analytical skills and ability to troubleshoot complex data-related issues.

Communication: Excellent communication and collaboration skills to work effectively with cross-functional teams and stakeholders.