Gcp Data Engineer

Role & responsibilities

Preferred candidate profile

Job Overview:
We are looking for a skilled GCP Data Architect with expertise in big data technologies like Spark, Kafka, Scala, Hadoop, and the Google Cloud Platform (GCP). The successful candidate will be responsible for designing, architecting, and implementing scalable and high-performance data solutions on GCP to support advanced analytics, real-time processing, and enterprise data integration. This role requires a deep understanding of both traditional and modern data architectures, cloud-native solutions, and hands-on experience with big data technologies.

Key Responsibilities:

• Data Architecture Design: Design, architect, and implement complex data pipelines and architectures on GCP for batch and real-time data processing.
• Big Data Solutions: Implement big data solutions using Apache Spark, Apache Kafka, Scala, and Hadoop frameworks, ensuring scalability, reliability, and optimal performance.
• Cloud Expertise: Leverage the full potential of Google Cloud Platform (GCP) services, such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and other data-related services.
• Data Integration: Architect and implement data integration solutions to connect various data sources (structured and unstructured) using Kafka and other streaming technologies.
• Data Governance & Security: Establish data governance frameworks, including data lineage, metadata management, data security, and compliance on GCP.
• ETL/ELT Pipelines: Design and develop robust ETL/ELT pipelines using Spark and other data tools to handle large-scale data ingestion, transformation, and storage.
• Performance Optimization: Continuously optimize data architectures, workflows, and pipelines for efficiency, cost, and performance improvements.
• Collaboration: Work closely with data engineers, data scientists, DevOps teams, and stakeholders to understand business needs and translate them into technical solutions.
• Documentation: Provide detailed documentation of data architectures, solutions, and processes, ensuring knowledge sharing within the team.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or related fields.
• 6+ years of hands-on experience in data architecture, big data processing, and cloud platforms.
• 3+ years of experience working with Google Cloud Platform (GCP) and its data services.
• Expertise in Apache Spark, Apache Kafka, Scala, and Hadoop.
• Strong experience with distributed data systems and real-time data streaming.
• Experience with CI/CD, infrastructure as code (IaC), and DevOps practices on GCP.
• Knowledge of data security, data governance, and compliance on cloud platforms.
• Proven experience with ETL/ELT pipelines, data modeling, and orchestration tools.

Perks and benefits