Sr AWS Data Engineer

Job Description

Position: Sr AWS Data Engineer
Location: Pune/ Gurgaon/ Hyderabad
Experience: 6+ years

Mode of Work : Hybrid

Responsibilities:

• Design, develop, and maintain large-scale data pipelines using AWS services such as Glue, Lambda, S3, and Redshift.
• Architect and implement data solutions in AWS Redshift for optimized data storage and retrieval.
• Build, automate, and monitor data workflows using AWS Step Functions and CloudFormation using templates.
• Set up and manage AWS Lake Formation and Glue Data Catalog for data lake environments.
• Develop and optimize ETL processes using Python and PySpark to process and transform large datasets.
• Implement data governance and security frameworks using AWS IAM and ensure compliance with organizational policies.
• Collaborate with data scientists, analysts, and stakeholders to understand data requirements and translate them into scalable solutions.
• Troubleshoot, optimize, and enhance existing data pipelines and architecture for performance and scalability.
• Implement best practices for data governance, security, and quality across the AWS infrastructure (PaaS).

Required Skills:

• 6+ years of experience in data engineering with a focus on AWS.
• Strong expertise in Python, PySpark, and SQL for building and optimizing ETL pipelines.
• In-depth knowledge of AWS Glue, Lambda, S3, Redshift, and AWS Step Functions.
• Hands-on experience with AWS CloudFormation using templates for infrastructure automation.
• Proficiency in AWS Lake Formation and Glue Data Catalog for data lake management.
• Strong understanding of IAM (Identity and Access Management) for securing AWS environments.
• Experience with performance tuning, data modeling, and query optimization in AWS environments.
• Advanced skills in implementing data security, governance, and compliance in cloud infrastructures.

Good to Have Skills:

• Experience with setting up CI/CD pipelines for data workflows.
• Knowledge of AWS cost optimization practices.
• Familiarity with additional AWS services such as Athena, DynamoDB, or RDS.