ob Title: Senior Big Data Engineer
Location: Chennai / Pune
Experience: 8+ Years
Job Description:
We are seeking an experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in Big Data technologies, distributed computing, and data processing frameworks, with expertise in Hadoop, Spark, Scala, and Cloudera. Proficiency in either Java or Python is essential.
Key Responsibilities:
- Design, develop, and implement scalable big data solutions using Hadoop, Spark, and related technologies.
- Build and optimize data pipelines and workflows to support business intelligence and analytics.
- Develop high-performance distributed applications using Scala, Java, or Python.
- Work with large-scale structured and unstructured datasets to perform data ingestion, transformation, and storage.
- Implement best practices for data governance, security, and performance tuning.
- Collaborate with cross-functional teams including data scientists, analysts, and DevOps engineers.
- Optimize and troubleshoot Hadoop ecosystem components for maximum efficiency.
- Deploy and manage Cloudera-based big data platforms.
- Automate data processing and ETL tasks to enhance operational efficiency.
Required Skills & Qualifications:
- 8+ years of experience in Big Data Engineering or a related field.
- Strong expertise in Hadoop ecosystem (HDFS, YARN, Hive, HBase, Oozie, Sqoop, etc.).
- Hands-on experience with Apache Spark for large-scale data processing.
- Proficiency in Scala, Java, or Python.
- Experience working with Cloudera or other enterprise big data platforms.
- Solid understanding of distributed computing principles and data architecture.
- Experience with data modeling, ETL pipelines, and real-time data processing.
- Familiarity with cloud services (AWS, Azure, or GCP) is a plus.
- Strong problem-solving and analytical skills.
- Ability to work independently and within a team in a fast-paced environment.
Preferred Qualifications:
- Cloudera/Hortonworks certification.
- Experience with streaming technologies like Kafka, Flink, or Storm.
- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines.
- Experience with NoSQL databases (Cassandra, MongoDB, etc.).
Employment Type: Full Time, Permanent
Read full job description