Senior Consultant - Tech Consulting - Data Engineer - Pune

We are seeking an experienced Spark Scala Developer to join our data engineering team

The ideal candidate will be responsible for developing and optimizing big data processing and analytics solutions using Apache Spark and Scala

You will work closely with data scientists, analysts, and other developers to transform complex data workflows into scalable and reliable systems

Key Responsibilities

Technical Skills

Technical skills in Design, build, and maintain scalable and high-performance data processing pipelines using Apache Spark and Scala

, develop ETL processes to ingest, transform, and store large volumes of data from various sources

Optimize Spark jobs for performance and cost efficiency, collaborate with cross-functional teams to understand data requirements and deliver solutions that meet business needs

Write clean, maintainable, and well-documented code, ensure data quality and implement data governance standards

Troubleshoot and resolve issues in the data pipelines

Stay up-to-date with the latest developments in big data technologies and contribute to the evolution of our data platform

Proven experience with Apache Spark and Scala programming, strong understanding of distributed computing principles and big data architectures, experience with Hadoop ecosystem components (HDFS, YARN, Hive, etc) is a plus

Qualifications

Soft Skills Strong communication skills, with an ability to collaborate and influence across teams

Proven experience of 5yrs in data engineering


Education Bachelor s degree in information technology, Computer Science or Management Information Systems or equivalent work experience

People with the ability to work in a collaborative manner to provide services across multiple client departments while following the commercial and legal requirements

You will need a practical approach to solving issues and complex problems with the ability to deliver insightful and practical solutions

We look for people who are agile, curious, mindful and able to sustain positive energy, while being adaptable and creative in their approach