Java Developer - Spark (8-15 yrs)

Job Description :

We are seeking a highly skilled and motivated Java Developer with experience in Apache Spark to join our dynamic team.

The successful candidate will possess a strong background in Java development and big data processing, coupled with a deep understanding of the architecture and capabilities of Spark.

Key Responsibilities :

• Design, develop, implement, and optimize large-scale data processing pipelines utilizing Java and Apache Spark.
• Write clean, efficient, and scalable Java code to support big data applications.
• Conduct data analysis, perform data transformations, and execute ETL processes on large datasets.
• Collaborate closely with data engineers, data scientists, and other stakeholders to gather and understand data requirements and translate them into technical solutions.
• Troubleshoot, debug, and enhance existing Spark applications to improve performance and functionality.
• Optimize Spark jobs by improving memory management, partitioning strategies, and caching mechanisms.
• Ensure data quality, consistency, and integrity across the data processing lifecycle.
• Work with cross-functional teams to design and implement data-driven solutions that meet business objectives.
• Continuously stay updated on the latest developments in big data technologies and industry best practices.

• A Bachelor's degree in Computer Science, Engineering, or a related field.
• Proven experience as a Java developer, with a strong understanding of Java 8 or later versions.
• Hands-on experience with Apache Spark, including core components such as Spark SQL, Spark Streaming, and Spark MLlib.
• Expertise in developing, deploying, and maintaining large-scale data processing applications.
• Strong knowledge of distributed computing principles and frameworks.
• Familiarity with big data technologies such as Hadoop, Kafka, and NoSQL databases.
• Experience with cloud platforms such as AWS, Google Cloud, or Azure is a plus.
• Excellent problem-solving abilities, with a keen attention to detail.
• Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams.

• Experience with Scala or Python is an advantage.
• Familiarity with containerization and orchestration tools such as Docker and Kubernetes.
• Understanding of CI/CD pipelines and DevOps practices.