Senior Data Engineer

Business Team:

The Chief Data and Analytics Office (CDAO) - Data Platforms is in place to realise a data-driven organisation for Commonwealth Bank Group. We do this by solving complex problems around data for the business and activating data for strategic and sustained competitive advantage to enhance the financial wellbeing of our customers in a safe, sound and secure way.

As we embark on our cloud journey, the team is focussed towards building robust data processing frameworks for the bank, enabling secure and seamless data processing for our tenants in simplified and automated way.

Impact Contribution:

As a data engineer, you will be part of a large cloud engineering cohort that are all aligned on executing the technology strategy, working alongside AWS Engineers and Cloud Architects to help create scalable and federated cloud native systems. We are establishing a greenfield landscape from ground up, partnering with our stakeholders to design, develop and deliver frameworks that underpin our ability to deliver enhanced data solutions with robust controls for our internal customers deriving value for Banks customers.

Bring your expertise in building scalable frameworks on cloud platform, with focus on automation and innovation.

Roles responsibilities:

• Design and implement the AWS Data Marketplace platform using AWS Data Zone, and related cloud-native services.

• Implement robust data governance practices, ensuring compliance with privacy, security, and regulatory standards.

• Collaborate with data architects and business stakeholders to define requirements and ensure the platform meets business needs.

• Develop automation frameworks for deployment, monitoring, and incident response to ensure platform reliability and performance.

• Optimize platform operations for cost efficiency and performance across services like S3, Lambda, Glue, Athena, and EMR.

• Ensure seamless integration of the data marketplace with internal and external data sources and applications.

• Develop and maintain API gateways and user-friendly interfaces for self-service data access and marketplace interactions.

• Troubleshoot platform issues and continuously improve platform reliability through monitoring and feedback loops.

• Mentor and guide junior engineers, fostering a culture of technical excellence and continuous learning.

Essential Skills:

• 5+ years of experience in designing and building platforms on AWS, with deep expertise in cloud-native architectures.
• Hands-on experience with AWS services such as Data Zone, AWS Marketplace, S3, IAM, Cloud Formation, API Gateway, Lambda, Glue, and Athena.
• Strong programming skills in Python, Java, or a similar language for automation and data engineering tasks.
• Proficiency in designing secure and scalable infrastructure using IAC tools like Terraform or Cloud Formation.
• Solid understanding of data governance, privacy frameworks, and compliance requirements.
• Experience building CI/CD pipelines and deploying scalable cloud solutions.
• Strong knowledge of REST APIs, microservices architecture, and event-driven systems.

Education Qualifications:

• Bachelor s degree in Engineering ( Computer Science/Information Technology)