Senior Data Engineer - Google Cloud Platform (8-12 yrs)

Role : Senior GCP Data Engineer.

Mandatory Skills : GCP, Bigdata,ETL,Data warehousing,Pyspark, BigQuery.

Experience : 8 -12 Years.

Location : Chennai and Hyderabad.

Position Overview :

We are seeking an experienced Senior GCP Data Engineer with 8+ years of expertise in ETL, Data Warehousing, and Data Engineering. The ideal candidate will have hands-on experience with GCP services, Data Lakehouse architecture, and Agile/SAFe methodologies, coupled with strong leadership and communication skills.

Qualifications :

- 8+ years of experience in ETL & Data Warehousing.

- Should have excellent leadership & communication skills.

- Should have strong working experience on Data Lakehouse architecture.

- Should have experience in developing Data Engineering solutions using GCP BigQuery, Cloud Storage, AirFlow, Dataflow, Cloud Functions, Pub/Sub, Cloud Run, etc.

- Should have built solution automations in any of the above ETL tools.

- Should have executed at least 2 GCP Cloud Data Warehousing projects.

- Should have worked at least 2 projects using Agile/SAFe methodology.

- Should Have experience in Pyspark and Teradata.

- Should have working experience on any DevOps tools like GitHub, Jenkins and Cloud Native, etc & on semi-structured data formats like JSON, Parquet and/or XML files & written complex SQL queries for data analysis and extraction.

- Depth understanding on Data Warehousing, Data Analysis, Data Profiling, Data Quality & Data Mapping.

- Should have global experience and been part of a team with at least 15+ members in a global delivery model.

- Should have experience in working with product managers, project managers, business users, applications development team members, DBA teams and Data Governance team daily to analyze requirements, design, development and deployment technical solutions.

Responsibilities :

- Analyze the different source systems, profile data, understand, document & fix Data Quality issues.

- Gather requirements and business process knowledge in order to transform the data in a way that is geared towards the needs of end users.

- Write complex SQLs to extract & format source data for ETL/data pipeline.

- Create design documents, Source to Target Mapping documents and any supporting documents needed for deployment/migration.

- Design, Develop and Test ETL/Data pipelines.

- Design & build metadata-based frameworks needs for data pipelines.

- Write Unit Test cases, execute Unit Testing and document Unit Test results.

- Deploy ETL/Data pipelines.

- Use DevOps tools to version, push/pull code and deploy across environments.

- Support team during troubleshooting & debugging defects & bug fixes, business requests, environment migrations & other adhoc requests.

- Do production support, enhancements and bug fixes.

- Work with business and technology stakeholders to communicate EDW incidents/problems and manage their expectations.

- Leverage ITIL concepts to circumvent incidents, manage problems and document knowledge.

- Perform data cleaning, transformation, and validation to ensure accuracy and consistency across various data sources.

- Stay current on industry best practices and emerging technologies in data analysis and cloud computing, particularly within the GCP ecosystem.

Education : B.Tech in Computer Science or related field.

Certifications : Google Cloud Professional Data Engineer Certification.