i
Capco Technologies
149 Capco Technologies Jobs
Capco - Big Data Engineer - ETL (3-5 yrs)
Capco Technologies
posted 2mon ago
Flexible timing
Key skills for the job
Key Responsibilities :
- Design, develop, and deploy data pipelines and ETL/ELT processes using AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS EMR, AWS Kinesis, and AWS Redshift.
- Build and maintain data lakes and data warehouses on AWS.
- Develop and implement data quality checks and monitoring systems.
- Troubleshoot and resolve data-related issues.
- Collaborate with data scientists and analysts to understand business requirements and translate them into technical solutions.
- Stay up-to-date with the latest advancements in AWS big data technologies.
- Participate in code reviews and contribute to the improvement of team processes.
- Mentor junior team members and provide technical guidance.
Qualifications :
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience as a Big Data Engineer or a similar role.
- Strong experience with AWS services, including but not limited to : AWS Glue AWS Lambda AWS S3 AWS EMR AWS Kinesis AWS Redshift AWS Athena
- Proficiency in Python and SQL.
- Experience with data warehousing and data lake architectures.
- Experience with data modeling and data quality best practices.
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration skills.
- Ability to work independently and as part of a team.
Preferred Qualifications :
- Experience with Spark and PySpark.
- Experience with containerization technologies such as Docker and Kubernetes.
- Experience with stream processing technologies such as Apache Kafka.
- Experience with data visualization tools such as Tableau or Power BI.
- AWS Certifications (e.g., AWS Certified Big Data - Specialty)
Functional Areas: Software/Testing/Networking
Read full job descriptionPrepare for Big Data Engineer roles with real interview advice