Senior Data Engineer

Senior Data Engineer (Serving NP/ Immediate Only) Work Location- Gurgaon Basic (Must Have): 4+ years of Big data processing experience using Spark Spark-SQL/ PYSpark 4+ years of Experience in developing and running data engineering pipelines, link raw client data with data model, conduct data assessment, perform data quality checks, and transform data usingETL tools. Excellent in data transformations, data modelling, and data validation activities, as well as configure applications to the client context. Experience in developing scripts to validate, transform, and load raw data using programming languages such asPython Self-driven with strong sense of ownership and eager to learn new tools and technologies like moderndata warehouse technology (Snowflake / AWS Redshift / Azure Synapse) Working Experience on multiple domain (Pricing, Marketing , Retail, Healthcare etc.) Can manage stakeholder expectation Good Experience on designing artifacts related to data solutions Experience on Implementing Data Jobs (Batch / Real time Streaming) Excellent skills on SQL and Advanced Level SQL. Excellent analytical skills Knows data and DWH modelling Optimization of pipelinesGood To Have: Experience on Cloud AWS/Azure will be a plus Experience on any visualization tool (Power BI / Tableau / Qlikview) Knowledge of NoSQL Database (HBase / Mongodb / Cosmos etc.) Experience on any of the scheduling tools Knowledge of Big Data Processing Cluster (Azure HDInsight / AWS EMR / Hadoop/ Databricks ) Key Responsibilities: Requirement Gathering at solution level, also can manage stakeholder expectation Good Experience on designing artifacts related to data solutions Knows how to design data collection template Design and develop data applications using selected tools and frameworks as per requirement Read, extract, transform, stage and load data using different tools and frameworks as per requirement Perform tasks such as writing scripts, web scraping, calling APIs, write SQL queries, etc Work closely with the engineering team to integrate your work into our production systems. Process structured/unstructured data into a form suitable for analysis. Support business decisions by performing ad hoc analysis as per need.