Snowflake Data Engineer

JOB OVERVIEW

The Snowflake Engineer will play a pivotal role in building and operationalizing the minimally inclusive consumer data necessary for the enterprise data and analytics initiatives following industry standard practices and tools. The primary responsibilities will be in building, managing and optimizing data pipelines and then moving these data pipelines effectively into production for key data and analytics consumers like business/data analysts, data scientists or any persona that needs curated data for data and analytics use cases across the enterprise in Snowflake envinronment integrated with Azure stack.

ROLES AND RESPONSIBILITIES

• Create, test, and implement enterprise-level apps with Snowflake
• Design and implement features for identity and access management
• Solve performance issues and scalability issues in the system
• Transaction management with distributed data processing algorithms
• Possess end to end ownership
• Build, monitor, and optimize ETL and ELT processes with data models
• Migrate solutions from on-premises setup to cloud-based platforms
• Understand and implement the latest delivery approaches based on data architecture
• Project documentation and tracking based on understanding user requirements
• Develop tools and libraries that will help analytics team members more efficiently interface with huge amounts of data.
• Analyze large, noisy datasets and identify meaningful patterns that provide actionable results.
• Develop and automate new enhanced imputation algorithms.
• Work with data lakes and databases including any aspect of administration or support required to maintain them

TECHNICAL COMPETENCIES (Knowledge, Skills & Abilities)

• Excellent at solving analytical problems using quantitative approaches.
• Expertise in Snowflake - data modelling, ELT using Snowflake SQL, implementing complex stored Procedures and standard DWH and ETL concepts
• Comfort manipulating and analyzing complex, high-volume, high-dimensionality data from varying sources
• Must have experience at one end to end implementation of Snowflake cloud data warehouse
• Strong programming background, Strong coding skills in one of the following: Python, Java, and Pyspark.
• Average knowledge of DevOps workflow
• Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, Big Data model techniques using Python
• Experience in handling critical incidents and user service requests and responsible for SLA adherence, tracking and reporting
• Good written and oral communication skills.
• Good interpersonal skills.
• Highly self-motivated and directed.
• Ability to effectively prioritize and execute tasks in a high-pressure environment
• Strong background in Software Engineering

EDUCATION AND EXPERIENCE

• Bachelors degree in computer science, mathematics, or statistics
• 5+ Years of of Data and Analytics experience with minimum 2+ years in Snowflake Cloud Data warehousing.
• Minimum 3 years of exeperince with Python/SQL
• Good understanding of Azure cloud
• Extensive experience in Snowflake - Virtual warehouse (Compute), Data modeling & storage, data loading/unloading and data sharing, SnowSQL (CLI) as well as Snowflake Internals and integrations, SnowPipe implementation, Snowflake security including readers and consumers accounts.
• SQL knowledge and hands on experience on complex queries writing using with Analytical functions, Troubleshooting, problem solving and performance tuning of SQL queries accessing data warehouse as well as Strong knowledge on stored procedures
• Design and build production data pipelines from ingestion to consumption with Python, Pyspark or Scala.
• Good knowledge on orchestration/scheduler tools, like Airflow
• In-depth understanding of Data Warehousing, ETL concepts and modeling structure principles
• Experience working with Snowflake Functions, hands on exp with Snowflake utilities, stage and file upload features, time travel, fail safe.
• Experience working with Semi-Structured and Unstructured data