Senior Data Engineer

Responsibilities include:

• Design, implement, and optimize end-to-end data pipelines for ingesting,

processing, and transforming large volumes of structured and unstructured data.

• Develop data pipelines to extract and transform data in near real time using cloud

native technologies.

• Implement data validation and quality checks to ensure accuracy and consistency.
• Monitor system performance, troubleshoot issues, and implement optimizations to

enhance reliability and ePiciency.

• Collaborate with business users, analysts, and other stakeholders to understand

data requirements and deliver tailored solutions.

• Document technical designs, workflows, and best practices to facilitate knowledge

sharing and maintain system documentation.

• Provide technical guidance and support to team members and stakeholders as

needed.

Desirable Competencies:

• 5+ years of work experience
• Proficiency in writing complex SQL queries on MPP systems (Snowflake/Redshift)
• Experience in Databricks and Delta tables.
• Data Engineering experience with Spark/Scala/Python
• Experience in Microsoft Azure stack (Azure Storage Accounts, Data Factory and

Databricks).

• Experience in Azure DevOps and CI/CD pipelines.
• Working knowledge of Python
• Feel comfortable participating in 2-week sprint development cycles