Senior Data Engineer - ETL/Python (12-15 yrs)

Job Description :

We are seeking a highly skilled Senior Data Engineer / Big Data Architect with extensive experience in AWS Glue and NoSQL databases. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, implementing scalable ETL processes, and managing large datasets efficiently.

Key Responsibilities :

- Develop and optimize ETL pipelines using AWS Glue and other AWS data services.

- Design and implement scalable data solutions for structured and unstructured data.

- Work with NoSQL databases (MongoDB, BigTable, Redis, Cassandra, HBase, Neo4j, CouchDB) to ensure high-performance data storage and retrieval.

- Architect data processing frameworks for real-time and batch data processing.

- Collaborate with cross-functional teams to understand business requirements and translate them into data solutions.

- Ensure data governance, security, and compliance best practices.

- Performance tuning of large-scale data processing jobs.

Required Skills & Experience :

- Primary Skill : AWS Glue - Extensive hands-on experience in ETL processing, data transformation, and orchestration using AWS Glue.

- Secondary Skill : NoSQL Databases - Proficiency in any of MongoDB, BigTable, Redis, Cassandra, HBase, Neo4j, CouchDB.

- Cloud Expertise - Strong knowledge of AWS services such as S3, Lambda, Redshift, Athena, and EMR.

- Programming & Scripting - Expertise in Python, PySpark, or Scala for data processing.

- Data Modeling & Architecture - Strong understanding of database design, indexing, partitioning, and performance tuning.

- Big Data & Distributed Systems - Hands-on experience with large-scale data platforms and distributed computing frameworks.

- Experience with Workflow Orchestration - Airflow, Step Functions, or similar tools.

- Strong problem-solving skills and the ability to optimize data workflows.