SDET with GenAI

Collaborate with business stakeholders to identify and define use cases
Design develop and execute automated tests using Python to ensure the functionality and performance of the product
Conduct manual testing where necessary to complement automated tests
Write complex SQL queries for test data generation and validation
Test and validate the product developed by the developers ensuring it meets the specified requirements
Stay informed and proficient in GenAI evaluation frameworks applying them effectively to assess the product
Perform rigorous testing of machine learning ML and deep learning DL components
Knowledge of Synthetic data generation techniques
Demonstrate in depth knowledge of evaluation based frameworks such as RAGAS Retrieval Augmented Generation for Automated Systems and a basic understanding of how RAG Retrieval Augmented Generation works
Define and implement testing strategies
Exhibit excellent communication and documentation skills to clearly report issues document test cases and articulate the results to technical and non technical stakeholders
Apply analytical skills to troubleshoot and resolve complex problems ensuring the highest quality standards are met

Skills and Qualifications
Proven experience in testing AI ML based solutions particularly generative AI GenAI
Proficiency in automated testing with Python
Familiarity with manual testing processes
Strong knowledge of SQL for data manipulation and testing purposes
Demonstrated experience with GenAI evaluation frameworks and testing of ML DL solutions
Expertise in prompt engineering for LLMs including question generation and evaluation metrics
Comprehensive knowledge of RAGAS and RAG frameworks
Experience with evaluating chatbot use cases and ensuring their effectiveness and relevance
Exceptional communication and documentation abilities
Strong analytical and problem solving skills.