Senior AI Engineer

Generative AI Pipeline Development:

• Design and implement efficient and scalable pipelines for data ingestion, processing, and transformation, tailored for Generative AI workloads.
• Orchestrate the flow of data between various AI services, databases, and backend systems within the Generative AI context.
• Build and maintain CI/CD pipelines for deploying and updating Generative AI services and pipelines.

Data and Document Ingestion:

• Develop and manage systems for ingesting diverse data sources (text, images, code, etc) used in Generative AI applications.
• Implement OCR and other preprocessing techniques to prepare data for use in Generative AI pipelines.
• Ensure data quality, consistency, and security throughout the ingestion process.

AI Service Integration:

• Integrate and manage external AI services (eg, cloud-based APIs for image generation, text generation, LLMs) into our Generative AI applications.
• Develop and maintain APIs for seamless communication between AI services and backend systems.
• Monitor and optimize the performance of integrated AI services within the Generative AI pipeline.

Retrieval Augmented Generation (RAG) Pipelines:

• Design and implement RAG pipelines to enhance Generative AI capabilities with external knowledge sources.
• Develop and optimize data retrieval and indexing strategies for RAG systems used in conjunction with Generative AI.
• Evaluate and improve the accuracy and relevance of RAG-generated responses in the context of Generative AI applications.

Large Language Model (LLM) Integration:

• Develop and manage interactions with LLMs through APIs and SDKs within Generative AI pipelines.
• Implement prompt engineering strategies to optimize LLM performance for specific Generative AI tasks.
• Analyze and debug LLM outputs to ensure quality and consistency in Generative AI applications.

Backend Services Ownership:

• Design, develop, and maintain backend services that support Generative AI applications.
• Ensure the scalability, reliability, and security of backend infrastructure for Generative AI workloads.
• Implement monitoring and logging systems for backend services and pipelines supporting Generative AI.
• Troubleshoot and resolve backend-related issues impacting Generative AI applications.

• Education: Bachelors or Masters degree in Computer Science, Artificial Intelligence, Machine Learning, or a related field.

Experience:

• 5+ years of experience in AI/ML development with a focus on building and deploying AI pipelines and backend systems.
• Proven experience in designing and implementing data ingestion and processing pipelines.
• Strong experience with cloud platforms (eg, AWS, Azure, GCP) and their AI/ML services.

Technical Skills:

• Expertise in Python and relevant AI/ML libraries.
• Strong understanding of AI infrastructure and deployment strategies.
• Experience with data engineering and data processing techniques.
• Proficiency in software development principles and best practices.
• Experience with containerization and orchestration tools (eg, Docker, Kubernetes).
• Experience with version control (Git).
• Experience with RESTful APIs and API development.
• Experience with vector databases and their application in AI/ML, particularly for similarity search and retrieval.

Generative AI Specific Skills:

• Familiarity with Generative AI concepts and techniques (eg, GANs, Diffusion Models, VAEs, LLMs).
• Experience with integrating and managing Generative AI services.
• Understanding of RAG pipelines and their application in Generative AI.
• Experience with prompt engineering for LLMs in Generative AI contexts.

Soft Skills:

• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.
• Ability to work in a fast-paced environment.

• Experience with OCR and document processing technologies.
• Experience with MLOps practices for Generative AI.
• Contributions to open-source AI projects.
• Strong experience with vector databases and their optimization for Generative AI applications