Senior Data Engineer

Senior Data Engineer (Serving NP/ Immediate Only)

Work Location- Gurgaon

Basic (Must Have):

• 4+ years of Big data processing experience using Spark Spark-SQL/ PYSpark
• 4+ years of Experience in developing and running data engineering pipelines, link raw client data with data model, conduct data assessment, perform data quality checks, and transform data using ETL tools.
• Excellent in data transformations, data modelling, and data validation activities, as well as configure applications to the client context.
• Experience in developing scripts to validate, transform, and load raw data using programming languages such as Python
• Self-driven with strong sense of ownership and eager to learn new tools and technologies like modern data warehouse technology (Snowflake / AWS Redshift / Azure Synapse)
• Working Experience on multiple domain (Pricing, Marketing , Retail, Healthcare etc.)
• Can manage stakeholder expectation
• Good Experience on designing artifacts related to data solutions    
• Experience on Implementing Data Jobs (Batch / Real time Streaming)
• Excellent skills on SQL and Advanced Level SQL.
• Excellent analytical skills  
• Knows data and DWH modelling
• Optimization of pipelines

Good To Have:

• Experience on Cloud AWS/Azure will be a plus
• Experience on any visualization tool (Power BI / Tableau / Qlikview) 
• Knowledge of NoSQL Database (HBase / Mongodb / Cosmos etc.)        
• Experience on any of the scheduling tools
• Knowledge of Big Data Processing Cluster (Azure HDInsight / AWS EMR / Hadoop/ Databricks )

Key Responsibilities:

• Requirement Gathering at solution level, also can manage stakeholder expectation
• Good Experience on designing artifacts related to data solutions
• Knows how to design data collection template  
• Design and develop data applications using selected tools and frameworks as per requirement
• Read, extract, transform, stage and load data using different tools and frameworks as per requirement
• Perform tasks such as writing scripts, web scraping, calling APIs, write SQL queries, etc
• Work closely with the engineering team to integrate your work into our production systems.
• Process structured/unstructured data into a form suitable for analysis.
• Support business decisions by performing ad hoc analysis as per need.