i
Quess
107 Quess Jobs
Big Data/Java Developer - Spark/Hadoop (3-8 yrs)
Quess
posted 3d ago
Flexible timing
Key skills for the job
Location- Mumbai, Bangalore, Gurgaon, Indore
Experience 3-8 yrs
Job Description
1. Proficient in Core JAVA (Multithreading and Collections framework)
2. Proficient in cleaning, transforming, and analyzing vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts.
3. Proficient in designing efficient data processing pipelines by writing suitable Spark batch and streaming Jobs involving various transformations and aggregations.
4. Proficient in Spark data structures and APIs based on RDDs, Dataframe and Datasets.
5. Proficient in Spark query tuning and performance optimization
6. Proficient in usage of Hadoop Distributed File System (HDFS)
7. Proficient in data formats such as JSON, ORC, Parquet, Avro, etc.
8. Desirable: Understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)
9. Desirable : Working with unstructured database (Mongo DB, HBase, Cassandra)
10. Desirable : Understanding and experience of working on SQL databases, such as Oracle, MySQL or Postgres
11. Desirable : Experience with Zookeeper, Hive, Sqoop and other utilities of Hadoop Ecosystem.
12. Must have strong troubleshooting skill in debugging the failure of Spark Jobs on production cluster.
Functional Areas: Software/Testing/Networking
Read full job descriptionPrepare for Java Developer roles with real interview advice