Capco - Data Engineer - Quality & Security (12-14 yrs)

Job Title : Data Engineer - AWS Cloud

Job Summary :

We are seeking an experienced Data Engineer to join our team, responsible for ensuring the design, development, quality, security, and governance of our data assets in the AWS Cloud. The successful candidate will work closely with Architects, and business stakeholders to develop and implement best data engineering practices

Key Responsibilities :

- Design and Development of robust data pipelines involving technologies like (not limited to) Kafka, Snowflake, AWS Glue, S3, Spark SQL, AWS Athena and Python using principles of ETL and Datawarehouse .

- Ensure data accuracy, completeness, and consistency across all data sources and systems.

- Collaborate with cross-functional teams to implement data product principles, promoting a decentralized approach to data ownership and accountability.

- Collaborate with security teams to implement data encryption, access controls, and compliance with regulatory requirements.

- Maintain comprehensive documentation for data assets, including data lineage, data definitions, and data quality rules.

- Implement and manage a data catalogue to enhance metadata visibility and accessibility.

- Oversee the end-to-end data lifecycle, ensuring compliance with regulatory and organizational policies.

- Facilitate processes for data creation, storage, usage, and deletion.

- Track data lineage, ensuring transparency and traceability of data transformations.

- Collaborate with data architects to design and implement data architectures.

- Utilize AWS Cloud services (e.g. , Glue, Lake Formation, EMR, S3) to manage and govern data.

- Identify areas for process improvement and implement changes.

Requirements :

- 12+ years of experience in data design and engineering data governance, data quality, or data management.

- Strong understanding of key data Engineering architecture involving cloud and on prem data transformation.

- Strong understanding of data governance principles, data modelling, and data architecture.

- Experience with AWS Cloud services implementation (Glue, Lake Formation, S3, etc.)

- Knowledge of data security and compliance regulations (GDPR, IFRS etc.)

- Excellent communication and stakeholder management skills