Data Engineer

Key Responsibilities

1. Snowflake Development:
• Design and implement data pipelines and workflows on Snowflake.
• Develop and optimize Snowflake databases, schemas, and tables.
• Manage Snowflake features like virtual warehouses, shared data, and materialized views.
2. Data Integration:
• Build and maintain ETL/ELT pipelines using tools like Talend, Informatica, Matillion, or equivalent.
• Integrate data from various sources such as APIs, databases, and flat files.
3. Performance Optimization:
• Monitor and optimize query performance in Snowflake.
• Implement partitioning, clustering, and caching strategies to enhance performance.
4. Data Modeling:
• Design logical and physical data models based on business requirements.
• Ensure data accuracy, consistency, and security across pipelines.
5. Collaboration:
• Work closely with data analysts, business stakeholders, and other engineers to deliver data solutions.
• Document processes, workflows, and architecture.

Required Skills

1. Technical Proficiency:
• 4+ years of experience in data engineering with at least 2 years working on Snowflake.
• Strong experience with SQL for querying and scripting.
• Knowledge of data warehousing concepts and methodologies.
2. Cloud Expertise:
• Familiarity with cloud platforms such as AWS, Azure, or GCP.
• Hands-on experience with Snowflake utilities like SnowSQL and SnowPipe.
3. ETL/ELT Tools:
• Experience with ETL tools like Matillion, Talend, or Informatica.
• Scripting experience in Python, Java, or Scala for custom ETL pipelines.
4. Other Tools:
• Proficiency with tools like DBT (Data Build Tool) is a plus.
• Experience with CI/CD pipelines for deploying data solutions.
5. Soft Skills:
• Strong problem-solving and analytical abilities.
• Excellent communication and teamwork skills.

Preferred Qualifications

• SnowPro Certification or equivalent credentials.
• Experience in Agile/Scrum development methodologies.
• Knowledge of data governance and compliance standards (e.g., GDPR, HIPAA).