AWS Data Engineer – Hadoop Migration

We are seeking an experienced AWS Principal Data Architect to lead the migration of Hadoop DWH workloads from on-premise to AWS EMR. As an AWS Data Architect, you will be a recognized expert in cloud data engineering, developing solutions designed for effective data processing and warehousing requirements of large enterprises. You will be responsible for designing, implementing, and optimizing the data architecture in AWS, ensuring highly scalable, flexible, secured and resilient cloud architectures solving business problems and helps accelerate the adoption of our clients data initiatives on the cloud.

Key Responsibilities:
Lead the migration of Hadoop workloads from on-premise to AWS-EMR stack.
Design and implement data architectures on AWS, including data pipelines, storage, and security.
Collaborate with cross-functional teams to ensure seamless migration and integration.
Optimize data architectures for scalability, performance, and cost-effectiveness.
Develop and maintain technical documentation and standards.
Provide technical leadership and mentorship to junior team members.
Work closely with stakeholders to understand business requirements, and ensure data architectures meet business needs.
Work alongside customers to build enterprise data platforms using AWS data services like Elastic Map Reduce (EMR), Redshift, Kinesis, Data Exchange, Data Sync, RDS , Data Store, Amazon MSK, DMS, Glue, Appflow, AWA Zero-ETL, Glue Data Catalog, Athena, Lake Formation, S3, RMS, Data Zone, Amazon MWAA, APIs Kong
Deep understanding of Hadoop components, conceptual processes and system functioning and relative components in AWS EMR and other AWS services. Good experience on Spark-EMR
Experience in Snowflake/Redshift
AWS system engineering aspects of setting up CI-CD pipelines on AWS using Cloudwatch, Cloudtrail, KMS, IAM IDC, Secret Manager, etc
Extract best-practice knowledge, reference architectures, and patterns from these engagements for sharing with the worldwide AWS solution architect community

Basic Qualifications:
10+ years of IT experience with 5+ years of experience in Data Engineering and 5+ years of hands-on experience in AWS Data/EMR Services (e.g. S3, Glue, Glue Catalog, Lake Formation)
Strong understanding of Hadoop architecture, including HDFS, YARN, MapReduce, Hive, HBase.
Experience with data migration tools like Glue, Data Sync.
Excellent knowledge of data modeling, data warehousing, ETL processes, and other Data management systems.
Strong understanding of security and compliance requirements in cloud.
Experience in Agile development methodologies and version control systems.
Excellent communication an leadership skills. Ability to work effectively across internal and external organizations and virtual teams.
Deep exp on AWS native data services including Glue, Glue Catalog, EMR, Spark-EMR, Data Sync, RDS, Data Exchange, Lake Formation, Athena,
AWS Certified Data Analytics Specialty.
AWS Certified Solutions Architect Professional.
Experience on Containerization and serverless computing.
Familiarity with DevOps practices and automation tools.
Experience in Snowflake/Redshift implementation is additionally preferred.

Preferred Qualifications:
Technical degrees in computer science, software engineering, or mathematics
Cloud and Data Engineering background with Migration experience.

Other Skills:
A critical thinker with strong research, analytics and problem-solving skills
Self-motivated with a positive attitude and an ability to work independently and or in a team
Able to work under tight timeline and deliver on complex problems.
Must be able to work flexible hours (including weekends and nights) as needed.
A strong team player