Cloud Big Data Engineer - Python/Hadoop/Hive (5-8 yrs)

We are hiring Cloud Big Data Engineer.

Location : Pune/Chennai/Mumbai/Bangalore/Hyderabad/Gurgaon.

Experience level : 5+ years.

Required Skills : GCP, BigData, Pyspark, Python, Spark, Kafka, Hadoop, Hive, Scala/Django, DFS Elastic search.

Key Responsibilities :

- Design, develop, and implement big data solutions on the Google Cloud Platform (GCP).

- Develop and maintain data pipelines for data ingestion, transformation, and loading into data warehouses and data lakes.

- Work with Apache Spark, Hadoop (HDFS, Hive), and other big data technologies to process and analyze large datasets.

- Develop and deploy data pipelines using tools like Apache Kafka.

- Utilize NoSQL databases such as ElasticSearch for data storage and retrieval.

- Develop and maintain data quality checks and ensure data integrity.

- Collaborate with data analysts, data scientists, and other stakeholders to understand business requirements and translate them into technical solutions.

- Optimize data pipelines for performance and scalability.

- Troubleshoot and resolve data-related issues.

- Stay up-to-date with the latest advancements in big data technologies and cloud computing.

Qualifications :

Education : Bachelor's degree in Computer Science, Engineering, or a related field.

Experience : 5+ years of experience in big data engineering, data warehousing, or a related field.

Technical Skills :

- Strong proficiency in Python and experience with frameworks like Django.

- In-depth knowledge of Apache Spark, Hadoop (HDFS, Hive), and other big data technologies.

- Experience with data ingestion, transformation, and loading tools.

- Experience with NoSQL databases, particularly ElasticSearch.

- Experience with cloud platforms, preferably GCP.

- Proficiency in Linux/Unix and shell scripting.

Preferred Skills :

- Experience with data streaming platforms like Kafka.

- Experience with data visualization and reporting tools (e.g., Tableau, Power BI).

- Experience with containerization technologies (Docker, Kubernetes).

- Knowledge of distributed systems concepts (load balancing, sharding, data distribution).

- Cloud certifications (e.g., Google Cloud Certified Professional Data Engineer)