Engineer III, Software Quality Assurance

Job Title: Generative AI - Test Automation Engineer

Function: Software Engineering SQA / SDET | Generative AI Testing, Backend Validation

How will you make an impact

We are looking for an outstanding Software Quality Assurance Engineer III. We are seeking a dedicated Generative AI Testing Specialist to join our innovative team. The ideal candidate will play a crucial role in evaluating the performance, accuracy, and reliability of our generative AI models. You will work closely with data scientists, engineers, and product managers to ensure that our AI solutions meet high standards of quality and effectiveness. Moreover, you will collaborate with various teams to communicate and handle more intricate tasks.

What will you do

• Design and implement comprehensive testing strategies for generative AI models, including but not limited to functionality, performance, and usability testing.
• Develop and maintain test cases, scripts, and testing frameworks specifically tailored for generative AI applications.
• Conduct performance evaluations and provide detailed reports on model outputs, identifying areas for improvement.
• Collaborate with cross-functional teams to define testing requirements and ensure alignment with project goals.
• Analyze and interpret model behavior and outputs, providing feedback to the development team for optimization.
• Stay updated with the latest advancements in AI and testing methodologies to continuously enhance testing processes.
• Participate in code reviews and contribute to the overall quality assurance of AI projects.
• Support Data analysts by providing prepped and clean data to enable accurate and efficient analysis.
• Support reporting and analysis functions that enable data-driven decision-making, supplying to the achievement of business goals.
• Work independently or with guidance as per team needs, demonstrating flexibility and adaptability.
• Actively participate in the evaluation and review of software requirements with Developers and Business System Analysts to define complete test plans and test scenarios.
• Create and compose clear and concise test summary reports and weekly status reports, offering valuable insights to the relevant parties.

Education

• Bachelors degree in computer science, Data Science, Engineering, or a related field.

Preferred Experience

• 7+ Proven experience in software testing, with a focus on AI/ML applications preferred.
• Familiarity with generative AI concepts and technologies (e.g., GANs, transformers).
• Proficiency in programming languages such as Python, Java, or similar.
• Strong analytical skills and attention to detail.
• Excellent communication and collaboration abilities.
• Experience with testing frameworks and tools (e.g., Selenium, PyTest).
• End-to-End Test Automation experience with frontend web apps and non-GUI applications (Backend, REST APIs, Data)
• Knowledge of code versioning tools like GitHub (CI-CD)
• Knowledge of conversational AI technologies, natural language processing (NLP), and an understanding of GenAI principles is highly desirable.
• Working knowledge of Oracle, MSSQL, AWS S3, and SQL IDEs such as SQL Developer/DB Visualizer/Toad.
• Ability to analyze and understand ETL workflows, ensuring the reliability and efficiency of data processes.
• Proficiency in Python programming language, enabling efficient data manipulation and analysis.
• Familiarity with cloud infrastructure like AWS or Azure and respective cloud storage systems like AWS S3 /Azure Blob.
• Experience in development models such as Agile or waterfall, ensuring successful project execution.
• Strong analytical skills, attention to detail, and ability to work both independently.
• Experience in backend validation of reports developed using tools like Tableau or POWER BI, ensuring accurate and reliable reporting.
• Strong experience in data analytical testing and a good understanding of the data model, giving to data-driven decision-making.
• Ability to work in a fast-paced work environment with minimal supervision, demonstrating self-motivation and accountability.
• Strong technical, problem-solving, project coordination, and analytical skills, ensuring successful project outcomes.
• Ability to work on multiple releases and projects simultaneously and to work in different time zones if required.

Good to have Skills

• Experienced in NFR testing like load & performance using JMeter/LoadRunner