Big Data Engineer - Java/Spark (6-12 yrs)

Key Responsibilities :

- Design and implement scalable and efficient data processing pipelines using Apache Spark and Java.

- Develop APIs and services for data ingestion, transformation, and aggregation to support analytics and machine learning models.

- Collaborate with data scientists, analysts, and cross-functional teams to understand data requirements and deliver innovative data solutions.

- Optimize Spark jobs for performance and cost efficiency across large datasets.

- Ensure high data quality and integrity by implementing robust testing and validation practices.

- Lead the adoption of best practices in code quality, CI/CD, and DevOps within the data engineering team.

- Mentor junior developers and contribute to knowledge sharing and team development.

Required Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- At least 7 years of software development experience, with a minimum of 5 years focused on big data processing using Apache Spark and Java.

- Proficient in Java programming and familiar with the Scala programming language.

- Strong experience in designing and implementing scalable data processing pipelines using Apache Spark and Java.

- Solid understanding of big data technologies and ecosystems (HDFS, YARN, Kafka, HBase, etc.).

- Experience with cloud platforms (AWS, Azure, or GCP) and understanding of cloud-native services for data processing.

- Familiarity with DevOps tools and practices, including Docker, Kubernetes, Jenkins, and Git.

- Excellent problem-solving skills, with the ability to tackle challenges in large-scale data environments.

- Strong communication and collaboration skills, with the ability to work effectively in a team and interface with technical and non-technical stakeholders.