Capco - Big Data Engineer - ETL (3-5 yrs)

Key Responsibilities :

- Design, develop, and deploy data pipelines and ETL/ELT processes using AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS EMR, AWS Kinesis, and AWS Redshift.

- Build and maintain data lakes and data warehouses on AWS.

- Develop and implement data quality checks and monitoring systems.

- Troubleshoot and resolve data-related issues.

- Collaborate with data scientists and analysts to understand business requirements and translate them into technical solutions.

- Stay up-to-date with the latest advancements in AWS big data technologies.

- Participate in code reviews and contribute to the improvement of team processes.

- Mentor junior team members and provide technical guidance.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

- 3+ years of experience as a Big Data Engineer or a similar role.

- Strong experience with AWS services, including but not limited to : AWS Glue AWS Lambda AWS S3 AWS EMR AWS Kinesis AWS Redshift AWS Athena

- Proficiency in Python and SQL.

- Experience with data warehousing and data lake architectures.

- Experience with data modeling and data quality best practices.

- Strong analytical and problem-solving skills.

- Excellent communication and collaboration skills.

- Ability to work independently and as part of a team.

Preferred Qualifications :

- Experience with Spark and PySpark.

- Experience with containerization technologies such as Docker and Kubernetes.

- Experience with stream processing technologies such as Apache Kafka.

- Experience with data visualization tools such as Tableau or Power BI.

- AWS Certifications (e.g., AWS Certified Big Data - Specialty)