Data Engineer

1. Job Description

We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company

2. Required Skills

1. Experience with Spark , RDD, Dataframe , Spark Streaming
2. Hands on experiencing in developing, maintaining CI/CD pipelines using Jenkins other tools
3. Experience with integration of data from multiple data sources at scale
4. Experience with various messaging systems, such as Kafka, Kinesis , SQS
5. Proficient understanding of distributed computing principles
6. Hands on experience with NoSQL databases
7. Good knowledge of Big Data querying engine, such as Hive, and Impala

8. Tuning , Management of Hadoop cluster, with all included services

3. Responsibilities and Duties

1. Selecting and integrating any Big Data tools and frameworks required to build scalable and fault tolerant data pipelines

2. Implementing Data ingestion pipelines for Batch and streaming data at petabyte scale

3. Tuning in the infrastructure to make better utilization

1. Requirements

1. BS/MS degree in Computer Science, Engineering or a related subject
2. Proven 2-8 working experience in languages Java, Scala/Pyspark for Data” pipeline development
3. Proficient in executing/tuning Hive queries for petabyte scale data
4. Experience in creating Data pipeline using “Big data technology tools”

5. Well versed with cloud concepts, knowledge of AWS/Azure will be a plus

What are the organizations that the candidate should have worked for ?

SLP, Devices , E-Commerce

Interested candidates, please share the updated CV at Nisha.Mohite@ril.com