AWS Data Engineer - ETL/Python (5-10 yrs)

AWS Data Engineer

Location : Bangalore.

- We are seeking a highly skilled AWS- Developer- with 6 years of experience to join our innovative team.

- The ideal candidate will have in-depth knowledge and hands-on experience with AWS- services, particularly PySpark, AWS- Glue, Lambda, DMS, S3, SQL, and DBT.

- The successful candidate will be responsible for designing, developing, and maintaining scalable and secure cloud-based solutions.

Key Responsibilities :

PySpark & AWS- Glue :

- Design and develop ETL processes using PySpark and AWS- Glue.

- Optimize data transformation and processing- workflows.

- AWS- Lambda.

- Create and manage Lambda functions for serverless applications and microservices.

AWS- DMS :

- Implement and manage data migration solutions using AWS- Database Migration Service (DMS).

AWS- S3 :

- Set up and maintain S3 buckets for efficient data storage, ensuring data integrity and security.

SQL & DBT :

- Write, optimize, and maintain complex SQL queries for data extraction and transformation.

- Utilize DBT (Data Build Tool) for data modeling, transformation, and pipeline management.

Collaboration & Communication :

- Collaborate with data engineers, analysts, and other stakeholders to gather requirements and deliver high-quality solutions.

- Participate in code reviews, design discussions, and provide technical guidance to team members.

Monitoring & Troubleshooting :

- Monitor the performance of AWS- services and applications.

- Identify and resolve issues to ensure the smooth operation of cloud-based solutions.

Required Skills and Qualifications :

Experience :

- 6 years of hands-on experience in AWS- development.

Technical Skills :

- Proficiency in PySpark and AWS- Glue.

- Strong experience with AWS- Lambda, DMS, and S3.

- Proficient in SQL and experienced with DBT (Data Build Tool).

- Familiarity with other AWS- services such as EC2, RDS, IAM, and CloudWatch.

- Proficiency in programming languages such as Python, Java, or Node.js.

- Experience with relational databases and data warehousing solutions.

- Strong analytical and problem-solving skills.

- Excellent verbal and written communication skills.

Preferred Qualifications :

- AWS- Certified Solutions Architect, Developer, or other relevant certifications.

- Experience with Agile/Scrum methodologies.

- Knowledge of CI/CD pipelines and version control systems (e.g, Git).