Senior Data Engineer

Experience - 8+ Years

Key Responsibilities: Design, develop and maintain robust, scalable and efficient data pipelines to move, transform and load data into Snowflake from various data sources, ensuring performance and cost optimization. Replicate and optimize SQL Server stored procedures in Snowflake, ensuring functional accuracy and performance in the cloud environment. Create and manage data transformations within Snowflake using DBT, building reusable and reliable data models aligned with business requirements. Implement and manage automated workflows using Prefect Scheduler to streamline ETL/ELT processes, ensuring reliability, monitoring and alerting for critical workflows. Design and maintain CI/CD pipelines using tools such as Jenkins or GitLab, automating testing, deployment and version control. Collaborate with cross-functional teams, including data scientists, analysts and business stakeholders, to understand data requirements and deliver impactful solutions, simplifying technical concepts for non-technical stakeholders. Use version control systems like GitLab to manage collaborative code development, branching and versioning while fostering best practices in code management. Continuously monitor, troubleshoot and optimize data pipelines for performance, scalability and compliance with SLAs. Develop reconciliation frameworks and implement automated migration validation processes to ensure data consistency and accuracy during cloud migrations. Maintain comprehensive documentation for data pipelines, architectures and processes to enable effective knowledge sharing.

Qualifications:

Must-Have: 8+ years of hands-on experience as a Data Engineer, with deep expertise in Snowflake, CI/CD practices and DevOps principles. Proven experience replicating and optimizing SQL Server stored procedures in Snowflake. Strong experience with reconciliation frameworks and automated migration validation for ensuring data consistency during cloud migrations. Demonstrated cloud migration expertise, including designing and optimizing scalable data processing solutions. Proficiency with CI/CD pipelines using Jenkins, GitLab or similar tools, with integration into DevOps workflows. Strong communication and interpersonal skills with the ability to collaborate effectively with technical and non-technical stakeholders. Proven ability to manage multiple projects simultaneously with a focus on quality and business value. A problem-solving mindset, attention to detail and the ability to thrive in a fast-paced, evolving environment.Good-to-Have: Proficiency in T-SQL for writing complex queries and stored procedures. Experience with DBT for building and managing transformations. Familiarity with Prefect for orchestrating and automating workflows. Hands-on experience with Azure Cloud and cloud-related tools. Python experience for data processing or orchestration tasks.Education:Bachelors degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experienceEvery individual comes with a different set of skills and qualities so even if you don t tick all the boxes for the role today, we urge you to apply as there might be a suitable/unique role for you tomorrow!Facebook | Twitter | LinkedIn | Instagram | Youtube