Senior AWS Data Engineer (4-6 yrs)

Job Description :

Responsibilities :

- Design, develop, and maintain ETL pipelines using AWS Glue, including crawlers, triggers, and metadata management.

- Utilize AWS Athena for interactive querying of data stored in S3.

- Manage and optimize data stored in S3 buckets.

- Design and implement data warehousing solutions using AWS Redshift.

- Develop data processing pipelines using PySpark and Python.

- Write and maintain efficient SQL queries to extract, transform, and load data.

- Integrate data pipelines with CI/CD workflows using tools like Github, Jenkins, and AWS DevOps.

- Document data pipelines and solutions using tools like JIRA and Confluence (Nice to have)

- Experience with Apache Iceberg for data management is a plus (not mandatory).

Qualifications :

- 4+ years of experience in software development or a related field.

- In-depth knowledge of AWS cloud services, including S3, Redshift, Glue, Athena, and related services.

- Strong understanding of data warehousing concepts and ETL processes.

- Proficiency in PySpark, Python, and SQL.

- Experience with CI/CD methodologies and tools (Github, Jenkins, AWS DevOps) is a plus.

- Excellent problem-solving, analytical, and critical thinking skills.

- Strong communication, collaboration, and teamwork abilities