Lead Java/Spark Developer (5-8 yrs)

Job Description :

- Design, develop, and maintain scalable and efficient data processing systems using Java and Apache Spark.

- Work with data scientists, analysts, and other engineering teams to implement data-driven solutions and optimize performance.

- Develop, test, and deploy Spark applications to process large volumes of data efficiently.

- Collaborate in the design of ETL pipelines to process batch and streaming data.

- Ensure data quality, integrity, and scalability by using advanced data processing techniques.

- Troubleshoot, debug, and optimize existing Spark applications and Java code for better performance.

- Work with cloud platforms such as AWS or Azure to deploy and manage big data applications.

- Participate in code reviews, mentoring junior developers, and promoting best practices within the team.

- Stay updated with emerging trends and technologies related to big data, Spark, and Java.

Required Skills & Qualifications :

- 5-8 years of strong experience in Java development, with a focus on backend services and distributed computing.

- Hands-on experience with Apache Spark, including development of batch and streaming applications.

- Strong understanding of Hadoop ecosystem (HDFS, YARN, Hive, etc.) is a plus.

- Proficient in designing and implementing ETL pipelines.

- Experience with Big Data technologies like HBase, Kafka, and others.

- Familiarity with data processing frameworks and tools such as Apache Flink, FaaS, or Apache Beam.

- Proficiency in working with relational and NoSQL databases.

- Strong problem-solving skills and ability to troubleshoot complex systems.

- Familiarity with Agile methodologies (Scrum, Kanban).

- Good understanding of Cloud technologies (AWS, Azure, or GCP).

- Excellent communication and teamwork skills.

Preferred Skills :

- Experience with Kubernetes and Docker for containerized Spark jobs.

- Knowledge of Graph Databases, Machine Learning, or AI technologies.

- Scala experience is a plus.