Data Engineer

Job Description :Data Engineer

Experience : 6 Yrs to 9 Yrs

Rel Expedience : 5 yrs

Notice Period : 15 days to Immediate

Data Engineer

We are seeking a highly skilled Data Engineer to join our team to execute a critical project around Point-of-Sale Data Management and Analytics. The ideal candidate will have a strong background in creating and maintaining robust and scalable ETL pipelines in the global warehouse.

Key Responsibilities

Building and operating large scale data infrastructure in production (performance, reliability, monitoring)

Designing, implementing, and debugging ETL pipelines

Thinking through long-term impacts of key design decisions and handling failure scenarios

Qualifications

Experience with ETL process (data extraction, data cleansing, data transformation, data monitoring, data modelling)

Experience with Microsoft Azure (Azure Data Factory, Azure Databricks, Azure Storage, Azure SQL Database, Azure Synapse)

Experience with Big Data systems (batch & stream processing, Spark, Hive)

Experience with two or more coding languages (including Python with pyspark, panadas)

Experience with SQL (Spark SQL, T-SQL),

Experience with code versioning (GIT)

Experience with various data formats (CSV, JSON, Parquet, Delta Lake)

Understanding of distributed systems concepts and principles (consistency and availability, liveness and safety, durability, reliability, fault-tolerance)

Eager to learn new things and passionate about technology