Data Engineer - Google Cloud Platform (4-10 yrs)

About the Role :

We are seeking a highly skilled and experienced GCP Data Engineer to join our growing team.

In this role, you will be responsible for designing, developing, and maintaining data pipelines and data warehousing solutions on the Google Cloud Platform (GCP).

You will work closely with data scientists, analysts, and business stakeholders to understand their data needs and translate them into robust and scalable data solutions.

Responsibilities :

- Design, develop, and maintain data pipelines and data warehousing solutions on GCP, including but not limited to :

1. Data ingestion from various sources (e., databases, APIs, cloud storage)

2. Data transformation and enrichment using tools like Spark, Pyspark, and Dataflow

3. Data loading and management in data warehouses (e., BigQuery, Spanner)

4. Data quality checks and monitoring

- Develop and maintain data pipelines using a variety of tools and technologies, including :

1. Cloud Dataflow

2. Cloud Pub/Sub

3. Cloud Storage

4. BigQuery

5. Spanner

6. Dataflow

- Work with data scientists and analysts to understand their data needs and translate them into effective data solutions.

- Collaborate with cross-functional teams to ensure data integrity, security, and compliance.

- Troubleshoot and resolve data-related issues.

- Stay up-to-date on the latest GCP data technologies and best practices.

Qualifications :

- 4-10 years of experience in data engineering or a related field.

- Strong experience in Big Data technologies like HDFS, Hadoop, Hive, Yarn, Pig, HBase, Sqoop, Flume.

- Working experience on Big Data services in a GCP cloud-based environment.

- Experience in Spark, Pyspark, Python or Scala, Kafka, Akka, core or advanced Java, and Databricks.

- Knowledge of how to create and debug Hadoop and Spark jobs.

- Experience with SQL and NoSQL databases.

- Experience with data modeling and data warehousing concepts.

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration skills.

- Experience with Agile methodologies.

Preferred Qualifications :

- Experience with machine learning and/or artificial intelligence.

- Experience with data visualization tools (e., Tableau, Power BI).

- GCP certifications