GCP Lead

GCP Lead Job Description: Expert Knowledge on GCP Data Analytics BigQuery, DataProc, Looker Ability to interact with business stakeholders and able to communicate effectively Support Google on daily basis during development and sit phase to review project progress, suggest improvements in execution and help to suggest bugs and its fixes. Create High Level weekly progress reports to show case progress and highlight issue areas and concerns that may result in project delays. Take code handover from Google to become primary to deliver successful UAT and Production deployments GCP Data Engineer: Key Responsibilities: 1. Data Pipeline Development: - Design, develop, and maintain robust and scalable data pipelines using GCP services such as Dataflow, Pub/Sub, Cloud Functions, and Cloud Composer. - Implement ETL (Extract, Transform, Load) processes to ingest data from various sources into GCP data warehouses like BigQuery. 2. Data Architecture: - Architect and optimize complex data storage solutions leveraging GCP services, including BigQuery, Cloud Storage, and Cloud Spanner. - Ensure data architecture is designed for scalability, reliability, and performance. 3. Data Integration and Management: - Develop solutions for integrating data from multiple on-premises and cloud sources. - Implement data quality and data governance frameworks to ensure data accuracy and reliability. 4. Collaboration and Communication: - Work closely with data scientists, analysts, and other engineers to understand data requirements and deliver optimal data solutions. - Provide technical leadership and mentorship to junior data engineers and other team members. 5. Monitoring and Optimization: - Monitor and troubleshoot data pipelines to ensure data availability and performance. - Optimize existing data processes for cost-efficiency and performance improvements. 6. Security and Compliance: - Implement and maintain data security measures in line with industry standards and regulatory requirements. - Ensure compliance with data privacy laws and corporate policies. 7. Documentation and Best Practices: - Document data engineering processes, architectures, and pipelines. - Promote best practices in data engineering and contribute to continuous improvement initiatives.