Data Engineer - Google Cloud Platform (5-8 yrs)

About the Role :

We are seeking a highly skilled and experienced GCP Data Engineer to join our team remotely.

You will play a crucial role in designing, developing, and maintaining robust and scalable data pipelines on the Google Cloud Platform (GCP).

Working closely with delivery managers and other stakeholders, you will contribute to building high-quality software solutions in an agile environment.

This is a 6-month contract position with the possibility of extension.

Key Responsibilities :

- Drive the development backlog, collaborating with delivery managers to define work, measure progress, and ensure timely delivery of high-quality software solutions within an agile framework.

- Design and develop high-performance data pipeline frameworks from scratch, encompassing data ingestion, transformation, storage, and reconciliation.

- Implement data ingestion processes from various systems, ensuring data integrity and accuracy.

- Focus on data quality and curation, establishing processes for data validation, cleansing, and enrichment.

- Perform data transformation and optimization for efficient storage and retrieval.

- Implement data reconciliation, monitoring, and control mechanisms to ensure data reliability and consistency.

- Support reporting models and other downstream application needs by providing accurate and readily available data.

- Create and maintain technical design documentation, data models, and performance-tuned applications.

- Collaborate effectively with cross-functional teams, demonstrating strong communication and coordination skills.

- Navigate ambiguity and thrive in a fast-paced environment with multiple stakeholders.

- Apply excellent structured thinking skills to break down complex, multi-dimensional problems.

Required Skills and Experience :

- 5+ years of experience in Data Engineering and Analytics.

- Strong understanding and hands-on experience with the following GCP services :

1. Google Cloud Storage (GCS)

2. BigQuery

3. Cloud Pub/Sub (Streaming)

4. Cloud Dataproc

5. Cloud Dataflow

6. Proficiency in Python and PySpark.

7. Experience with Kafka.

- Strong SQL skills, including writing complex queries and stored procedures.

- Proficiency in shell scripting.

- Experience with data modeling and performance tuning.

- Excellent communication and collaboration skills.

Qualifications : Bachelor's Degree (Engineering/Computer Science preferred but not required); or equivalent experience