GCP Data Engineer - Consultant

The person will work on a variety of projects in a highly collaborative, fast-paced environment. The person will be responsible for software development activities of KPMG, India. Part of the development team, he/she will work on the full life cycle of the process, develop code and unit testing. He/she will work closely with Technical Architect, Business Analyst, user interaction designers, and other software engineers to develop new product offerings and improve existing ones. Additionally, the person will ensure that all development practices are in compliance with KPMG s best practices policies and procedures. This role requires quick ramp up on new technologies whenever required.

Bachelor s or master s degree in computer science, Information Technology, or a related field.

Key Responsibilities:

• Data Integration : Integrating various data sources into a unified data warehouse or data lake using tools like Cloud Dataflow, Cloud Dataproc, and Cloud Pub/Sub.
• Data Pipeline Development : Creating and managing data pipelines to automate the movement and transformation of data.
• Data Storage Management : Selecting and managing appropriate storage solutions such as BigQuery, Cloud Storage, and Cloud SQL.
• Data Transformation : Transforming raw data into a format that can be easily analyzed using tools like Dataflow and Dataprep.
• Performance Optimization : Monitoring system performance, identifying bottlenecks, and implementing solutions to improve efficiency.
• Security and Compliance : Ensuring data security and compliance with regulations through best practices for data encryption, access control, and auditing.
• Collaboration and Communication : Working closely with other teams to ensure the integrity and usability of data.
• Proficiency in BigQuery, Cloud Dataflow, Dataproc, Pub/Sub, Cloud Functions, Cloud SQL, and Cloud Storage.
• Experience with Python and SQL for data processing and querying.
• Knowledge of Apache Beam for data processing.
• Strong understanding of cloud-native architectures and optimization strategies within GCP
• Familiarity with data governance frameworks and best practices for data security and compliance.