Big Data Engineer - Google Cloud Platform (5-8 yrs)

Position : Sr Big Data Engineer (GCP)

Experience : 5 to 8 Yrs

Location : Remote

Shift Time : EST Shift (5:30 PM to 2:30 AM)

Position Overview :

We are seeking a highly skilled Senior Big Data Engineer with extensive experience in Big Data technologies, including Hadoop, Hive, PySpark, and expertise in Google Cloud Platform (GCP) and BigQuery. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics.

This role requires excellent data analysis skills and the ability to architect solutions for cloud-based environments, with a focus on extracting, processing, and analyzing data from diverse sources.

Key Responsibilities :

- Design, develop, and optimize Big Data pipelines and workflows using Hadoop, Hive, PySpark, and Google BigQuery.

- Architect data solutions to facilitate smooth data migration and processing between on-premises systems and Google Cloud Platform (GCP).

- Create and manage Unix Shell Scripts for process automation and data handling tasks.

- Perform in-depth data analysis and work comfortably with large datasets to derive actionable insights.

- Collaborate effectively with cross-functional teams to understand data requirements and translate them into scalable data solutions.

- Ensure data quality, integrity, and security across all stages of the data lifecycle.

- Stay updated with the latest industry trends and emerging technologies to continuously improve data solutions.

Mandatory Qualifications :

- Minimum of 8 years of industry experience working with Big Data technologies.

- Proficiency in Hadoop, Hive, and PySpark for data processing.

- Strong experience with Unix Shell Scripting.

- Expertise in SQL and extracting data from multiple sources.

- Demonstrated experience in data analysis, specifically querying and analyzing large datasets on Hadoop HDFS using Hive and Spark.

- Proven experience in architecting Big Data pipelines and data solutions.

- Proficiency in Google BigQuery and migrating data pipelines to GCP.

- Exceptional communication skills for clear requirement gathering and collaboration.

Preferred Qualifications :

- Google Cloud Platform (GCP) certification.

- Professional experience with cloud hosting platforms, with a preference for GCP.