GenAI & Platform Test Lead

Job Description:

As a GenAI/Platform Test Lead, you will play a crucial role in ensuring the quality, reliability, and performance of our language models and related software systems. You will be responsible for designing and executing comprehensive test plans, identifying and documenting defects, and collaborating with cross-functional teams to deliver high-quality products. Your expertise in testing methodologies and tools will contribute to the successful development and deployment of GenAI platforms.

Key Responsibilities:

• Test Planning & Strategy - Collaborate with data scientists and developers to understand requirements and define test plans, test cases, and test scenarios accordingly. Design testing strategies specifically targeting fine-tuning processes, including validation of data and result.
• Test Execution - Execute manual and automated tests to test platform, ensuring that model adjustments improve performance metrics such as accuracy, fluency, and relevance. Perform in-depth analysis of model outputs and compare against baseline metrics to identify improvements and areas for further optimization.
• Defect Identification & Reporting - Identify and document issues related to fine-tuning processes, model performance, and data quality using issue tracking systems. Collaborate with data scientists and developers to troubleshoot issues, conduct root cause analysis, and implement corrective actions.

Skills & Tools

• Test Management Tools - Proficiency in using test management tools such as TFS, TestRail, PractiTest, or Xray for organizing test cases, tracking test execution, and generating test reports.
• Automation Testing Tools/Frameworks - Experience with automation testing tools/frameworks such as Selenium, PyTest, TensorFlow Extended (TFX), or Hugging Face Transformers for automating fine-tuning validation processes.
• Collaboration Platforms - Experience with collaboration platforms such as Jira, Confluence, or Slack for managing tasks, documenting findings, and facilitating communication within the team.