Data Engineer with AWS Glue

Position: Data Egnineer with AWS Glue

Location: Hyderabad, India

No of Open Positions: 1

Duration: Full Time with Caspex Tech LLP

Mandatory Skills

• Pyspark - 4+ yrs
• Python - 2+ Yrs
• AWS(Glue, Lambda, Step-Functions, Crawler, Athena) 2+ yrs
• SQL 2+
• Familiar with Linux System and Unix Shell Scripting.
• Strong problem-solving abilities in handling large-scale datasets.
• Ability to collaborate with cross-functional teams and communicate effectively with stakeholders.
• Good to have experience in ETL and ETL migration from On Prem to AWS.

Job Description:

• Working with Developers & UAT users to understand architecture and requirement specifications to document expected test results
• Experience in at least one end to end testing of data migration (ETL or data pipeline) to the cloud (preferably AWS)
• Automation experience working with technologies: must have knowledge of testing "Python, Glue PySpark, S3, Lambda etc."
• SQL and ability to write complex queries to validate data between source and target
• Create and maintain data scenarios to support test plans and test cases
• Good understanding of working in an Agile/Scrum environment and ability to document test plan, test cases, test results with expected and actual
• Analyzing test results, documenting and tracking bugs to closure
• Ability to work in every phase of development process
• Experience in obtaining test results sign-off from business users/stakeholders, with timely follow up/follow through