Recro.io - Lead Big Data Engineer - Hadoop/Spark (8-13 yrs)

The role involves building and managing Big Data Pipelines to handle large structured datasets for scalable analytics solutions. The primary focus will be on selecting optimal tools and implementing, maintaining, and monitoring them while integrating with the company's architecture.

Responsibilities :

• Work closely with Product Management & Engineering leadership to build the right solution.
• Participate in design discussions to select, integrate, and maintain Big Data tools and frameworks.
• Develop distributed processing systems for cleansing, processing, and analyzing large datasets using Akka and Spark.
• Critically review existing data pipelines and propose improvements.
• Takes initiative and works independently as a Senior Individual Contributor on multiple products.
• Develop highly scalable Big Data pipelines with at least 3+ years of experience.

• Worked with Spark, Akka, Storm, Hadoop, and various file formats.
• Experience with ETL tools like Apache NiFi and Airflow.
• Strong coding skills in Java or Scala, with knowledge of design patterns.
• Proficiency in Git, Gradle/Maven/SBT.
• Solid understanding of OOP, data structures, algorithms, profiling, and optimization.

• Strong verbal and written communication skills.
• Ability to work under pressure and manage multiple projects.
• Passion for learning, problem-solving, and troubleshooting.