Lead Data Engg (Snowflake)

The Snowflake Lead Data Engg will play a pivotal role in building and operationalizing the minimally inclusive consumer data necessary for the enterprise data and analytics initiatives following industry standard practices and tools. The primary responsibilities will be in building, managing and optimizing data pipelines and then moving these data pipelines effectively into production for key data and analytics consumers like business/data analysts, data scientists or any persona that needs curated data for data and analytics use cases across the enterprise in Snowflake envinronment integrated with Azure stack.

ROLES AND RESPONSIBILITIES

• Architect, create, maintain, manage, and optimize data pipelines as workloads move from development to production for specific use cases.
• Responsible for using innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity.
• Assist with renovating the data management infrastructure to drive automation in data integration and management.
• Develop algorithms and predictive models to solve critical business problems.
• Develop tools and libraries that will help analytics team members more efficiently interface with huge amounts of data.
• Analyze large, noisy datasets and identify meaningful patterns that provide actionable results.
• Develop and automate new enhanced imputation algorithms.
• Work with data lakes and databases including any aspect of administration or support required to maintain them

TECHNICAL COMPETENCIES (Knowledge, Skills & Abilities)

• Excellent at solving analytical problems using quantitative approaches.
• Comfort manipulating and analyzing complex, high-volume, high-dimensionality data from varying sources
• Must have experience at least two to three end to end implementation of Snowflake cloud data warehouse
• Strong programming background, Strong coding skills in one of the following: Python, Java, and Pyspark.
• Strong knowledge of DevOps workflow, cloud-native platforms (containers, Kubernetes, serverless, etc.) and tools (such as Git), and infrastructure as code (IaC) tools (such as CloudFormation or Terraform)
• Expertise in Snowflake advanced concepts like setting up resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, Zero copy clone, time travel and understand how to use these features
• Expertise in deploying Snowflake features such as data sharing, events and lake-house patterns
• Hands-on experience with Snowflake utilities, SnowSQL, SnowPipe, Big Data model techniques using Python
• Experience in Data Migration from RDBMS to Snowflake cloud data warehouse
• Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modelling)
• Experience with data security and data access controls and design
• Strong ability to collaborate with a variety of roles and harmonize the relationship between business and IT.
• Strong passion for empirical research and for answering hard questions with data.
• Flexible analytic approach that allows for results at varying levels of precision.
• Good written and oral communication skills.
• Good interpersonal skills.
• Highly self-motivated and directed.
• Ability to effectively prioritize and execute tasks in a high-pressure environment
• Strong background in Software Engineering