Data Engineer / Sr. Engineer

Position Summary:

We are seeking a Data Engineer / Sr. Engineer within our dynamic Data Solutions team. This role is pivotal in bridging the gap between teams, delivering innovative data solutions that empower informed decision-making and drive strategic initiatives. Focus on rigorous QA, effective tools, and optimized processes to enhance data quality across projects.

Your Role Responsibilities and Duties:

• Quality Assurance & Quality Control: Institute and uphold rigorous practices to ensure that all data products delivered by the team meet established quality standards.
• Toolset Identification & Process Improvement: Focus on identifying the appropriate toolset and instituting new processes, including standalone QA/QC methods and enhanced developer workflows aimed at improving overall data quality.
• Data Profiling: Utilize mid-level SQL skills to profile data effectively, including performing complex joins and using Common Table Expressions (CTEs) to analyse data sets.
• Version Control Proficiency: Demonstrate fluency in GitHub workflows and best practices to facilitate collaborative development and version control.
• Test Plan Creation: Develop comprehensive multi-level test plans employing various methodologies (manual, automated, end-to-end, unit tests, data quality scripts, etc.) to ensure project goals are met.
• Test Case Development: Create test cases for in-development projects based on product specifications and requirements gathered from team discussions.
• Regression Testing: Design and execute regression test cases for existing data products, informed by code reviews and direct observations of system performance.

Required Skills and Qualifications:

• SQL and Mastery: Exceptional skills in writing, optimizing, and debugging complex SQL queries.
• Python Governance: Experience with python, governance tool (e.g. Atlan, Alation, Collibra) or data quality tool (e.g. Great Expectations, Monte Carlo, Soda) will be added advantage
• DBT Proficiency: In-depth knowledge of DBT for data transformation, modelling, and workflow orchestration.
• Data Warehousing: Strong understanding of data warehousing concepts and methodologies, particularly in cloud environments.
• Snowflake Expertise: Hands-on experience with Snowflake, including data modelling, query optimization, and system integration.
• Analytical Thinking: Proven ability to analyse large, complex datasets and derive actionable insights.
• Problem Solving: Adept at identifying issues and implementing effective solutions with attention to detail.
• Collaboration & Communication: Excellent interpersonal and communication skills to work effectively with cross-functional teams.

What we Offer:

• Bootstrapped and financially stable with high pre-money evaluation.
• Above industry renumerations.
• Additional compensation tied to Renewal and Pilot Project Execution.
• Additional lucrative business development compensation.
• Firm building opportunities that offer stage for holistic professional development, growth, and branding.
• Empathetic, excellence and result driven organization. Believes in mentoring and growing a team with constant emphasis on learning.