Architect - Data Engineer

Responsibilities

Lead the migration and modernization of data platforms, moving applications and pipelines to Azure-based solutions.

• Actively contribute to code development in projects and services.
• Manage and scale data pipelines from internal and external data sources to support new product launches and ensure high data quality.
• Develop automation and monitoring frameworks to capture key metrics and operational KPIs for pipeline performance.
• Implement best practices around systems integration, security, performance, and data management.
• Collaborate with internal teams, including data science and product teams, to drive solutioning and proof-of-concept (PoC) discussions.
• Develop and optimize procedures to transition data into production.
• Define and manage SLAs for data products and operational processes.
• Prototype and build scalable solutions for data engineering and analytics.
• Research and apply state-of-the-art methodologies in data and Platform engineering.
• Create and maintain technical documentation for knowledge sharing.
• Develop reusable packages and libraries to enhance development efficiency.

Qualifications

Bachelors degree in Computer Science, MIS, Business Management, or related field

• 10 + years experience in Information Technology
• 4 + years of Azure, AWS and Cloud technologies
• Experience in data platform engineering, with a focus on cloud transformation and modernization.
• Strong knowledge of Azure services, including Databricks, Azure Data Factory, Synapse Analytics, and Azure DevOps (ADO).
• Proficiency in SQL, Python, and Spark for data engineering tasks.
• Hands-on experience building and scaling data pipelines in cloud environments.
• Experience with CI/CD pipeline management in Azure DevOps (ADO).
• Understanding of data governance, security, and compliance best practices.
• Experience working in an Agile development environment.
• Prior experience in migrating applications from legacy platforms to the cloud.
• Knowledge of Terraform or Infrastructure-as-Code (IaC) for cloud resource management.
• Familiarity with Kafka, Event Hubs, or other real-time data streaming solutions.
• Experience with lagacy RDBMS (Oracl, DB2, Teradata)
• Background in supporting data science models in production.