Senior Data Engineer - Hadoop/Hive/Spark (3-8 yrs)

Job Description

- Strong Data Engineering Profile

Mandatory (Experience 1) :

- Must have 5+ YOE in Data Engineering using data transformation tools like HDFS, YARN, Map-Reduce, Hive, Kafka, Spark, Airflow, Presto, etc.

Mandatory (Experience 2) :

- Must have worked on large distributed architectures using any of Kafka, Spark, Hive, Hadoop

Mandatory (Experience 3) :

- Must have handled projects involving at least 100 GB of data.

Mandatory (Core Skills) :

- Expertise in Apache Spark (RDDs, data frames, Spark tuning) and PySpark.

Mandatory (Data Handling) :

- Familiarity with data formats like Parquet, Avro, and NoSQL databases.

Mandatory (Tech Stack) :

- Proficiency with distributed systems and big data technologies such as HDFS, YARN, Kafka, Hive, MapReduce, Hadoop, etc.

Mandatory (Company) :

- Mid-sized Product Companies or Analytics-heavy companies