Senior Data Engineer

preferredRole & responsibilities

Responsibilities include: • Design, implement, and optimize end-to-end data pipelines for ingesting, processing, and transforming large volumes of structured and unstructured data. • Develop data pipelines to extract and transform data in near real time using cloud native technologies. • Implement data validation and quality checks to ensure accuracy and consistency. • Monitor system performance, troubleshoot issues, and implement optimizations to enhance reliability and efficiency. • Collaborate with business users, analysts, and other stakeholders to understand data requirements and deliver tailored solutions. • Document technical designs, workflows, and best practices to facilitate knowledge sharing and maintain system documentation. • Provide technical guidance and support to team members and stakeholders as needed.

Preferred candidate profile

• 8+ years of work experience • Proficiency in writing complex SQL queries on MPP systems (Snowflake/Redshift) • Experience in Databricks and Delta tables. • Data Engineering experience with Spark/Scala/Python • Experience in Microsoft Azure stack (Azure Storage Accounts, Data Factory and Databricks). • Experience in Azure DevOps and CI/CD pipelines. • Working knowledge of Python • Feel comfortable participating in 2-week sprint development cycles.

Perks and benefits

Immediate Joiners preffred