Data Engineer- Spark with Scala- Immediate joiners

Job Description: Big Data Engineer- Immediate joiners only

Location: Bangalore

Experience Level:6+ years

We are seeking a highly skilled Big Data Engineer with expertise in distributed systems, the Hadoop ecosystem, and advanced programming techniques. The ideal candidate will possess strong technical knowledge, thought leadership, and a passion for creating scalable, high-performance systems.

Key Responsibilities:

• Design, develop, and optimize distributed systems and Big Data solutions.
• Implement and maintain batch and streaming pipelines using Scala and Spark.
• Leverage experience in the Hadoop ecosystem, including technologies like Hive, Oozie, and Kafka.
• Build and maintain CI/CD pipelines to ensure efficient code deployment and integration.
• Apply design patterns, optimization techniques, and locking principles to enhance system performance and reliability.
• Scale systems and optimize performance through effective caching mechanisms.
• Demonstrate strong computer science fundamentals, logical reasoning, and problem-solving abilities.
• Collaborate with cross-functional teams to influence and drive innovative solutions.
• Provide technical leadership, mentorship, and presentations to guide team members and stakeholders.

Required Qualifications:

• Proven experience working with distributed systems and Big Data technologies.
• Proficiency in Scala, Spark, and related tools.
• In-depth understanding of design patterns and system optimisation principles.
• Hands-on experience with batch and streaming data pipelines.
• Familiarity with CI/CD pipelines and modern software development practices.
• Strong understanding of system scaling and caching techniques.
• Excellent communication and presentation skills, with the ability to inspire and influence others.

Preferred Qualifications:

• Scala, spark(preferably scala should be an experienced spark exposure
• Good understanding of Spark Architecture including Spark Core, Spark SQL, data Frames, Spark Streaming, Driver Node, Worker Node, stages, Executors and Tasks.
• Should have experienced with Hadoop ecosystem-HDFS, hive (must), Oozie, sqoop etc.
• Should have prior experience working with big data ingestion from Various sources handling TB of data.
• Excellent Knowledge of SQL and coding skills.
• Experience with scripting languages such as Shell.
• Effective working in a team as well as working independently.
• Experience of working in Agile delivery.

We would love to hear from you if you are passionate about working on complex systems and leading the charge in cutting-edge Big Data solutions.