Data Engineer - AWS/Spark (5-8 yrs)

We are seeking a skilled and motivated Data Engineer to join our team. The ideal candidate will have expertise in AWS, Spark, Python, AWS Lambda, Glue, and Airflow, with 6years of relevant experience. This role is critical for developing and maintaining scalable data pipelines and ensuring efficient data integration across our platforms.

Key Responsibilities :

- Design, build, and maintain data pipelines using AWS, Spark, Python, AWS Lambda, Glue, and Airflow.

- Develop and deploy scalable ETL processes to integrate data from various sources.

- Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.

- Optimize and troubleshoot data workflows to ensure performance and reliability.

- Implement best practices for data governance, security, and compliance.

- Document data integration processes and maintain technical documentation.

- Continuously evaluate new tools and technologies to enhance data engineering capabilities.

Qualifications :

- Bachelor's degree in Computer Science, Information Technology, or a related field.

- 6 to 7 years of relevant experience in data engineering.

- Proficient in AWS services, Spark, Python, AWS Lambda, Glue, and Airflow.

- Strong understanding of ETL processes and data integration techniques.

- Experience with big data technologies and processing large datasets.

- Excellent problem-solving skills and attention to detail.

- Strong communication and collaboration skills.

- Ability to work independently and as part of a team in a fast-paced environment.