Google Cloud Platform & BigQuery Developer - Data Pipeline (5-7 yrs)

Key Responsibilities :

GCP & BigQuery Development :

- Design, implement, and optimize data pipelines and solutions in Google Cloud Platform (GCP), primarily focusing on BigQuery.

- Develop and maintain BigQuery data models and queries, ensuring performance optimization and cost management.

- Leverage BigQuery for large-scale data analysis, aggregations, and reporting.

Data Pipeline Development :

- Build and maintain data pipelines using Python, ensuring seamless data flow and transformations between systems.

- Utilize Apache Airflow or Google Composer for orchestration and automation of data workflows.

SQL Expertise :

- Write complex SQL queries to perform various data manipulations, transformations, and aggregations.

- Optimize and tune SQL queries for performance in BigQuery.

Collaboration & Documentation :

- Collaborate with cross-functional teams to understand data requirements and deliver high-quality data solutions.

- Ensure detailed documentation of work, including data pipeline design, query optimization strategies, and process workflows.

Solution Design & Architecture :

- Design scalable and efficient GCP solutions for data storage, transformation, and analytics.

- Work closely with architects and business teams to translate requirements into technical specifications and design robust data systems.

Continuous Learning & Best Practices :

- Stay up to date with the latest GCP technologies and best practices, contributing to the continuous improvement of the data engineering processes.

- Follow industry best practices for data security, data governance, and performance optimization in cloud environments.

Required Skills & Experience :

Experience :

- Minimum 5 years of hands-on experience working with Google Cloud Platform (GCP) and BigQuery.

Data Pipeline Expertise :

- Strong proficiency in Python for building and managing data pipelines, with a focus on data processing and automation.

SQL Skills :

- Expertise in writing complex SQL queries for data manipulation, aggregation, and analysis.

Cloud Orchestration :

- Experience with Apache Airflow or Google Composer for orchestrating and automating data workflows.

GCP Certifications (Good to Have) :

- GCP Solution Architect certification is highly preferred.

Collaboration & Communication :

- Ability to work collaboratively with team members and stakeholders from diverse backgrounds and skill sets.

Attention to Detail & Documentation :

- Strong attention to detail, ensuring all work is well-documented and follows best practices.

Preferred Qualifications :

Certifications :

- GCP Solution Architect - Certified is highly preferred.

Other Skills :

- Familiarity with other GCP services such as Cloud Storage, Cloud Functions, Dataflow, and Pub/Sub is a plus