AWS AI Data Engineer

This role is part of one of the workstreams of Project Acuity, specifically the PASD Data Platform a centralized web application for internal PASD users across the Recruitment Business. The platform supports marketing and operational use cases, with a focus on building a patient-level database to enhance future reporting capabilities and external stakeholder engagement.

Timings: Minimum 6 hours overlap with US timings (2 PM to 11 PM IST)

We are seeking an experienced AWS AI Data Engineer to join our dynamic team. You will be responsible for designing, developing, and optimizing data architectures that support AI and Machine Learning (ML) workflows. The ideal candidate will have extensive experience in integrating large-scale datasets, building scalable and automated data pipelines, and working with advanced ML frameworks and tools. Additionally, expertise in AWS ETL services (e.g., AWS Glue, Lambda, Data Pipeline) for efficient data processing and integration is required.

• Design and implement scalable AI/ML-driven data architectures on AWS.
• Develop automated data pipelines to integrate large-scale datasets from multiple sources.
• Work with machine learning frameworks (e.g., TensorFlow, PyTorch) to build and optimize AI models.
• Implement data classification techniques, including identification of PII data entities.
• Develop retrieval-augmented generation (RAG) workflows and agent-based AI models.
• Optimize LLM (Large Language Models) outputs using indexing and vector store techniques.
• Utilize AWS services (e.g., SageMaker, Comprehend, Entity Resolution) to address AI and data engineering challenges.
• Deploy and manage ML models at scale using AWS cloud infrastructure.
• Develop ETL workflows using AWS Glue, Lambda, and Data Pipeline to ensure seamless data integration.
• Implement and manage core AWS services including IAM, VPC, EC2, S3, RDS, Lambda, CloudWatch, and CloudTrail.

1. Proficiency in programming languages such as Python, Scala, or similar.
2. Hands-on experience with machine learning frameworks (e.g., TensorFlow, PyTorch).
3. Expertise in data classification, including identifying PII data entities.
4. Strong knowledge of retrieval-augmented generation (RAG) workflows.
5. Deep understanding of indexing techniques to improve LLM performance.
6. Hands-on experience with AWS AI services (e.g., SageMaker, Comprehend, Entity Resolution).
7. Experience in deploying and managing ML models at scale using AWS infrastructure.
8. Strong problem-solving and analytical skills for AI-driven data engineering.
9. Expertise in AWS ETL services (AWS Glue, Lambda, Data Pipeline) for data processing and integration.
10. Experience with AWS cloud services including IAM, VPC, EC2, S3, RDS, Lambda, CloudWatch, CloudTrail.

1. Experience in data privacy compliance (especially related to PII data).
2. Familiarity with advanced data indexing techniques, vector databases, and technologies that improve AI/ML outputs.