Technical Lead - Big Data (8-10 yrs)

Job Title : Tech Lead - Big Data, Digital Business

Experience : 8+ Years (5+ years in Data Technologies, 3+ years in AWS Cloud Services)

Budget : 1.6 LPM

Work Mode : Work from Office (WFO)

Location : Gurgaon

Job Description :

Key Responsibilities :

Data Engineering Leadership :

- Lead and manage Big Data engineering initiatives, ensuring robust and scalable solutions.

- Design and implement data pipelines, data integration, and data processing frameworks using advanced technologies.

AWS Cloud Expertise :

- Utilize AWS Cloud Services for data storage, processing, and analytics.

- Architect, develop, and manage scalable and cost-effective cloud-based solutions.

OLAP Systems :

- Leverage ClickHouse or other OLAP systems for high-performance analytics and reporting.

- Optimize query performance and data retrieval on OLAP systems.

Programming Proficiency :

- Develop scalable and efficient solutions using Java or Scala.

- Implement scripts in Python for automation and data workflows (preferred).

Distributed Data Processing Frameworks :

- Work with frameworks like Spark (Core, Streaming, SQL), Storm, or Flink to handle large-scale distributed data processing.

- Optimize frameworks for batch and real-time data processing tasks.

Collaboration and Communication :

- Collaborate with cross-functional teams to translate business requirements into technical solutions.

- Provide technical guidance and mentorship to junior team members.

Performance Optimization :

- Identify and resolve performance bottlenecks in data systems and workflows.

- Continuously improve data processing pipelines for efficiency and reliability.

Required Skills :

- Big Data Expertise: 5+ years in data technologies and distributed systems.

- AWS Cloud Services : Proficiency with data-related AWS services such as S3, EMR, Redshift, Glue, or Athena.

- OLAP Systems : 3+ years of experience with ClickHouse or other OLAP tools.

- Programming Languages : Strong expertise in Java and Scala; Python is a plus.

- Distributed Frameworks : Hands-on experience with Spark, Storm, or Flink for distributed data processing.

- Strong problem-solving skills and ability to handle large-scale data challenges.

Preferred Qualifications :

- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

- Certifications in AWS Cloud or Big Data technologies are advantageous.

- Experience working in Agile/Scrum environments.