GCP Data Engineer (5-8 yrs)

GCP Data Engineer (5-8 Years Experience)

Are you passionate about leveraging cutting-edge technologies to build scalable, transformative data solutions? We are seeking an experienced and dynamic GCP Data Engineer to join our innovative team in Bangalore. In this role, you will work on both real-time and batch data systems, ensuring the delivery of high-performance, reliable, and impactful solutions. If you thrive in a fast-paced environment and have a flair for solving complex data challenges, this opportunity is for you!

About the Role :

As a GCP Data Engineer, you will play a pivotal role in designing, developing, and implementing advanced data engineering frameworks. Using Google Cloud Platform (GCP) services and big data tools, you will create scalable solutions to process and analyze large volumes of data. Your work will empower our organization to make data-driven decisions, drive innovation, and achieve strategic objectives.

Key Responsibilities :

1. Data Engineering & Framework Development :

- Design and deploy robust data ingestion and transformation pipelines using GCP tools like DataProc, Cloud Composer, and BigQuery.

- Develop solutions for real-time and batch data processing using tools such as Apache Spark, PySpark, and Hadoop.

- Build efficient data storage and retrieval mechanisms that meet performance and scalability requirements.

2. Workflow Orchestration :

- Leverage orchestration tools such as Apache Airflow and Oozie to manage and automate complex workflows.

- Implement monitoring and alerting mechanisms to ensure the seamless execution of data pipelines.

3. Programming & Data Processing :

- Utilize Python and SQL to develop data transformation scripts and optimize queries.

- Integrate data streams in real time using platforms like Apache Kafka, ensuring data consistency and low latency.

4. Data Governance & Open Source Integration :

- Operationalize open-source tools and frameworks while adhering to stringent data governance, compliance, and security policies.

- Ensure data accuracy, quality, and reliability through effective validation processes.

5. Collaboration & Stakeholder Engagement :

- Partner with data analysts, data scientists, and business stakeholders to understand data requirements and deliver solutions aligned with business objectives.

- Provide technical guidance and share best practices with team members to foster a culture of continuous learning.

Key Skills and Expertise :

1. GCP Expertise : In-depth experience with DataProc, Cloud Composer, and BigQuery for managing large-scale data projects in a cloud environment.

2. Big Data Tools : Strong proficiency in big data technologies like Apache Spark, PySpark, Hadoop, and Hive for data processing and analysis.

3. Workflow Orchestration : Hands-on experience with tools like Apache Airflow or Oozie to streamline workflows effectively.

4. Advanced Programming : Expertise in Python, SQL, and real-time data streaming platforms like Kafka.

5. Enterprise-Level SDLC : Proven track record of implementing Software Development Life Cycle (SDLC) in enterprise-level projects over 5-7 years.

Qualifications :

Education : Bachelor's degree in Computer Science, Information Technology, or a related field.

Experience : 5-8 years of experience in data engineering, with a strong focus on GCP services and big data ecosystems.

What We Offer :

- Work on cutting-edge technologies with a forward-thinking team.

- A collaborative and inclusive work culture in the heart of Bangalore.

- Opportunities for professional growth, training, and development.

- Competitive salary and comprehensive benefits package.

Why Join Us?

At our company, we believe in the transformative power of data. As a GCP Data Engineer, you will contribute to shaping our data strategy and play a critical role in driving innovation and business success. Join us to make a real impact while growing your career in a supportive and dynamic environment.

Ready to transform data into actionable insights?