Project Description:We are seeking a highly skilled Senior Snowflake Data Engineer to join our team. The ideal candidate will have extensive experience with Snowflake, DBT, PySpark, and Airflow to design, develop, and manage complex data pipelines, ETL processes, and cloud-based data warehousing solutions. This role is perfect for someone who thrives in a fast-paced environment and can work both independently and collaboratively with cross-functional teams to support data modernization initiatives.
Responsibilities:Key Responsibilities: • Data Pipeline Design & Development: Build and manage robust, scalable, and efficient data pipelines in Snowflake using DBT, PySpark, and Airflow. • Data Modeling: Design and optimize Snowflake data models for performance, scalability, and easy reporting. • ETL/ELT Implementation: Implement advanced ETL/ELT processes to ingest and transform data from multiple sources into Snowflake using DBT, PySpark, and other tools. • Automation & Orchestration: Leverage Apache Airflow for scheduling, monitoring, and orchestrating complex data workflows. • Collaboration: Work closely with data analysts, data scientists, and business stakeholders to understand data requirements and deliver optimal solutions. • Performance Optimization: Continuously optimize Snowflake queries, storage, and compute costs. • Data Governance: Ensure data quality, security, and compliance in accordance with organizational standards and industry best practices. • Documentation: Maintain up-to-date documentation for data processes, system configurations, and workflow designs.
Mandatory Skills Description:Experience: • Minimum of 5+ years of experience in data engineering. • 3+ years of hands-on experience with Snowflake, DBT, PySpark, and Airflow.
Required Skills: • Snowflake: Advanced experience in developing, optimizing, and managing Snowflake environments (schemas, roles, query tuning). • DBT: Proficiency with DBT (Data Build Tool) for data transformation and modeling in Snowflake. • PySpark: Strong experience in developing ETL jobs and data transformations using PySpark. • Airflow: Hands-on experience with Apache Airflow for job orchestration and scheduling. • SQL: Expertise in writing complex SQL queries for data analysis and reporting. • Cloud Technologies: Experience with cloud platforms (AWS, Azure, or GCP) and related services. • Scripting & Automation: Proficiency in Python and shell scripting for automation. • Data Warehousing: Strong understanding of data warehouse concepts, OLAP, and big data architecture.
Preferred Qualifications: • Experience with CI/CD pipelines for data deployments. • Experience with data lake integration using Snowflake. • Cloud Certification: AWS, GCP, or Azure data certifications are a plus. • Experience with other data tools: Kafka, Spark, or similar big data technologies.
Education: • Bachelors or Master’s degree in Computer Science, Data Engineering, or a related field.