Cloud Big Data Engineer - Hadoop/Hive/Spark (5-8 yrs)

About the Role :

We are seeking a highly skilled and experienced Cloud Big Data Engineer to join our growing team. In this role, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines and applications on the cloud.

You will work with large datasets, leveraging big data technologies to extract meaningful insights and drive business decisions.

Key Responsibilities :

- Hands-on experience with core big data technologies: Hadoop, Hive, HDFS, Spark (PySpark), and related ecosystem components.

- Develop and optimize data pipelines using Spark for data ingestion, transformation, and analysis.

- Strong understanding of cloud platforms, with a focus on Google Cloud Platform (GCP) is preferred.

- Experience with cloud-based data services (i.e., Google Cloud Storage, BigQuery, Dataflow).

- Experience with data ingestion, transformation, and loading (ETL/ELT) processes.

- Proficiency in data modeling and data warehousing concepts.

- Strong proficiency in Python and related frameworks (i.e., Django).

- Experience with shell scripting (Bash, etc.)

- Working knowledge of file formats such as JSON, Parquet, AVRO, and ORC.

- Experience with job scheduling and debugging in a production environment.

- Understanding of distributed systems concepts like load balancing, sharding, and data distribution.

Qualifications :

Experience : 5+ years of experience in Big Data engineering.

Technical Skills :

- Strong proficiency in Python, Spark (PySpark), Hadoop (HDFS, Hive).

- Experience with GCP or other cloud platforms.

- Working knowledge of data warehousing concepts.

- Experience with data ingestion and ETL processes.

Education : Bachelor's degree in Computer Science, Engineering, or a related field.

Other Skills :

- Excellent problem-solving and analytical skills.

- Strong communication and interpersonal skills.

- Ability to work independently and as part of a team