Data Engineer - PySpark (6-12 yrs)

Job Title : Data Engineer

Experience : 6+ Years

Location : Chennai, Bangalore, Gurgaon

Work Mode : Hybrid

We are seeking an experienced Data Engineer to design, develop, and optimize near real-time and streaming data pipelines. If you have expertise in PySpark, Apache Spark, Kafka, and cloud-based big data processing, we want to hear from you!

Key Responsibilities :

- Design, develop, and maintain real-time & streaming data pipelines

- Utilize Amazon EMR, Apache Spark, Apache NIFI for data processing

- Develop and optimize ETL workflows & storage solutions

- Implement Kafka frameworks (kSQL, Mirror Maker, etc.) for data streaming

- Collaborate with cross-functional teams to enhance data engineering solutions

- Ensure pipeline performance, scalability, and reliability

Required Skills :

- Proficiency in PySpark (Python API for Apache Spark)

- Experience with Kafka frameworks (kSQL, Mirror Maker)

- Strong programming skills in Python, Groovy, or Java

- Expertise in ETL design, data structures & storage solutions

- Hands-on experience with Amazon EMR, Apache Spark, Apache NIFI

Preferred Qualifications :

- Experience with big data frameworks & cloud platforms

- Familiarity with database design & optimization

- Strong analytical & problem-solving skills