Senior Platform Engineer

As a Senior Platform Engineer, you will work closely with the Platform Lead on the design, development, and deployment of our data platform on Google Cloud Platform (GCP). You will work closely with cross-functional teams to ensure the efficient and secure handling of data. Your expertise in Python, DBT, and SQL will be critical in driving our data strategy forward. Additionally, you will bring advanced visualisation skills (Tableau, Looker Studio) and experience with DevOps support (CI/CD - Azure DevOps), debugging, code reviews, and change control.

Key Responsibilities:

Design & Development:

Architect and implement scalable and secure data pipelines on GCP.

Utilise Python, DBT, and SQL to build, maintain, and optimise ETL processes.

Develop data models, schemas, and data warehouse solutions.

Implement advanced GCP services (GCP Composer, Cloud Run, BigQuery, Vertex AI, VPC Service Control, Cloud Function).

Collaboration & Communication:

Collaborate with data scientists, analysts, and other stakeholders to understand data requirements.

Communicate project status, risks, and issues to senior management.

Foster a culture of continuous improvement and innovation within the team.

Operations & Maintenance:

Monitor and optimise the performance of the data platform.

Troubleshoot and resolve issues related to data pipelines and infrastructure.

Conduct root cause analysis for failures and incidents.

Visualization & Reporting:

Develop and maintain data visualisations and dashboards using Tableau or Looker Studio.

DevOps & Support:

Support and build CI/CD pipelines and version control (Azure DevOps).

Perform debugging, code reviews, and ensure change control processes are followed.

Leadership & Strategy:

Provide technical leadership and guidance on best practices for data & cloud engineering

Qualifications:

Experience:

Minimum of 5 years of experience in cloud engineering or data engineering.

Proven experience with Google Cloud Platform (GCP).

Expertise in Python programming.

Proficiency in DBT (Data Build Tool) and SQL.

Advanced understanding of SQL.

Experience with GCP services like GCP Composer, Cloud Run, Cloud Function BigQuery, Vertex AI.

Skills:

Strong understanding of data warehousing concepts and practices.

Experience with CI/CD pipelines and version control (e.g., Azure DevOps).

Excellent problem-solving skills and attention to detail.

Strong technical leadership and communication skills.

Advanced Python coding skills.

Strong analytical skills.

Self-sufficient attitude or quickly able to obtain necessary information.

Good to Have:

Security and Compliance: Strong understanding of best practices in cloud security and incident response.

Scalability and Performance: Expertise in design and implement scalable data architectures to efficiently handle large volumes of data.

Cost Management: Experience in monitoring and managing GCP costs, ensuring efficient use of resources and budget adherence.

Terraform: Proficiency in Terraform for infrastructure as code (IaC) to automate the provisioning and management of GCP resources.