AWS Glue Developer - ETL/Python (5-10 yrs)

Title : AWS Glue Developer. 100%

Remote : Fulltime.

Qualifications :

- Design, develop, and implement ETL pipelines using AWS Glue.

- Create and manage AWS Glue crawlers, jobs, and workflows.

- Develop data transformation logic using Python and PySpark within AWS Glue.

- Work with Amazon S3 for data storage and retrieval.

- Utilize AWS Lambda for serverless data processing and automation.

- Design and optimize data models for Amazon Redshift.

- Write complex SQL queries for data analysis and manipulation.

- Monitor and troubleshoot ETL pipeline performance and data quality issues.

- Implement data validation and error handling mechanisms.

- Collaborate with data engineers, data scientists, and other stakeholders.

- Document ETL processes and data flows.

- Stay up-to-date with the latest AWS Glue and data engineering trends.

Skills :

- Proficient in creating crawlers, jobs, and workflows.

- Experience in managing data storage and retrieval.

- Ability to develop serverless functions for data processing.

- Experience in designing and optimizing data models.

- Strong knowledge of SQL for data querying and manipulation.

- Proficiency in Python for data transformation and scripting (PySpark experience is a plus).

- ETL principles and best practices.

- Data warehousing concepts.

- Data quality and data governance.

- Version control (Git).

Exp : 5+ yrs