Thoucentric - AWS Data Engineer - ETL/PySpark (4-7 yrs)

We are looking for an experienced AWS Data Engineer to design, develop, and maintain data solutions using core AWS services. The ideal candidate will have hands-on experience with tools like Amazon S3, Redshift, AWS Glue, and DynamoDB, and will build scalable, efficient, and secure data pipelines and architectures. The role also requires strong expertise in PySpark, SQL, and workflow orchestration tools like Airflow.

Key Responsibilities

1. Data Pipeline Development

- Develop and manage ETL/ELT workflows using AWS Glue and PySpark to process large datasets.

- Automate data workflows using Apache Airflow and other orchestration tools.

2. Data Storage and Management

- Architect and manage data storage in Amazon S3 , ensuring performance, cost-efficiency, and security.

- Create and optimize Amazon Redshift clusters for data warehousing and analytics workloads.

- Design scalable NoSQL solutions using Amazon DynamoDB for real-time data needs.

3. Compute and Serverless

- Build and deploy serverless solutions using AWS Lambda for event-driven data processing.

- Configure and manage virtual machine instances using Amazon EC2 for custom data processing tasks.

4. Security and Monitoring

- Implement fine-grained access controls with AWS IAM to ensure data security.

- Set up monitoring, logging, and alerts using AWS CloudWatch for proactive system health management.

5. Data Integration and Transformation

- Create efficient SQL queries to handle data transformations and analytics.

- Integrate and process structured and unstructured data from multiple sources.

- Design data models and implement them in Redshift or DynamoDB for optimized query performance.

6. Collaboration and Optimization

- Collaborate with data scientists, analysts, and stakeholders to gather requirements and deliver solutions.

- Continuously optimize data processing workflows to improve performance and reduce costs.

Required Skills and Qualifications :

- Core AWS Expertise

- Strong knowledge of Amazon S3 , Redshift , AWS Glue , Lambda , DynamoDB , EC2 , IAM , and CloudWatch .

Technical Proficiency :

- Hands-on experience with PySpark for big data processing.

- Proficient in writing complex SQL queries for data manipulation and analysis.

- Expertise in workflow orchestration using Apache Airflow or similar tools.

Experience :

- 4+ years in data engineering with a focus on AWS technologies.

- Experience in designing scalable, fault-tolerant data pipelines.

- Familiarity with data modeling, ETL/ELT, and data warehousing principles.

Soft Skills :

- Strong problem-solving and analytical skills.

- Excellent communication and collaboration abilities.

- Ability to manage priorities in a fast-paced environment

Benefits :

- What a Consulting role at Thoucentric will offer you ?

- Opportunity to define your career path and not as enforced by a manager

- A great consulting environment with a chance to work with Fortune 500 companies and startups alike.

- A dynamic but relaxed and supportive working environment that encourages personal development.

- Be part of One Extended Family. We bond beyond work - sports, get-togethers, common interests etc.

- Work in a very enriching environment with Open Culture, Flat Organization and Excellent Peer Group.

- Be part of the exciting Growth Story of Thoucentric!