Spark & Scala Developer - Big Data Technologies (5-8 yrs)

Job Description : Spark and Scala Developer

We are seeking a skilled Spark and Scala Developer to join our dynamic team. The ideal candidate will have hands-on experience in developing and optimizing data processing pipelines and possess strong expertise in big data technologies.

Key Responsibilities :

- Develop, test, and deploy scalable data processing applications using Scala , Spark , and related big data technologies.

- Design and implement data pipelines to process structured and unstructured data.

- Write optimized and efficient code in Scala and Python/PySpark- for data transformation and analysis.

- Work with SQL and NoSQL databases to manage and query large datasets.

- Integrate applications within the Hadoop ecosystem- (HDFS, Hive, HBase, etc.).

- Collaborate with cross-functional teams to define, design, and deliver new features.

- Troubleshoot and optimize performance issues in large-scale distributed systems.

- Provide technical guidance on data architecture and infrastructure requirements.

Required Skills :

- Proficiency in Scala coding with a solid understanding of functional programming concepts.

- Extensive experience with Apache Spark for data processing and analytics.

- Hands-on experience in Python/PySpark for data engineering tasks.

- Proficiency in querying and working with SQL/NoSQL databases (e.g., MongoDB, Cassandra).

- Familiarity with the Hadoop ecosystem and big data technologies.

- Prior experience in Java development (preferred).

- Strong problem solving skills and the ability to optimize complex data workflows.

Qualifications :

- Bachelor's degree in Computer Science, Engineering, or related field.

- 3+ years of experience in big data development and distributed systems.

- Excellent communication and collaboration skills.

This role offers the opportunity to work on cutting-edge technologies and challenging big data problems in a fast-paced environment.