Lead Big Data Engineer - Spark/Scala (10-20 yrs)

Location : Bangalore, Marathahalli, WFO 3 days a week

Top skills which are non negotiable - Databricks, Spark, Scala

Number of Rounds - 3

Urgency of requirement (how soon we are looking to hire) - Immediate

The successful candidate will be :

- Big Data engineer having extensive coding experience in Spark & Scala.

- He/She should be able to work well within a product-based organization (product mindset) and with a team comprising of developers & functional experts on a critical customer-facing product.

- Has prior experience of at least 2 years as a PO, or lead business analyst and is comfortable with technical products.

- Should have handled one e2e implementation of creating a data pipeline using spark and in-depth knowledge of spark performance tuning.

- Good to have experience in microservices-based architecture.

- Autonomous, motivated and self-driven.

- The candidate should have some prior experience in managing direct customer communication and expectations management and ability to handle ambiguity.

- The candidate should have strong stakeholder management skills to manage dependencies across multiple teams and lead discussion with multiple stakeholders.

- Support continuous improvement by investigating alternatives and technologies and presenting these for architectural review.

- Ready to work with a DevOps mindset with other engineers.

- Candidate should be able to understand Application architecture and align with overall product functionality and vision.

- Excellent English verbal and written communication skills.

- Strong analytical and Conceptual Thinking to perform efficient root cause analysis and drive resolutions.

- Airline industry knowledge is a strong positive.

Technical Skill :

- 10+ years of experience in analysis, design, development, documentation, implementation, and testing of software products using Spark, Scala and Big data tech stacks

- At least 6+ years of working with Big Data Tech stacks : Databricks, Kafka, Spark, and experience on Azure cloud platform.

- Good to have experience with NoSQL Data using MongoDB/Cassandra/HBase etc.

- Good to have experience in microservices-based architecture.

- Sound Knowledge of OOPs concepts (Class loading, Memory Management, Transaction management, Multithreading, Garbage collection, Performance optimization), Data structures, and Design Patterns.

- Sound experience with Streaming data using Apache Kafka and processing it in real time with optimum performance. Experience in Messaging architecture using MQ is a plus.

- Designing and developing high-volume, low-latency applications for mission-critical systems and delivering high availability and performance.