Senior Associate

Design, develop, and maintain automated test
frameworks and scripts for chat interface applications
utilizing LLMs, ensuring comprehensive coverage of
all functionalities and user scenarios.
Understand Retrieval Augmented Generation(RAG)
strategy to improve testing for knowledge-
intensive tasks by retrieving relevant documents or
data that can be used as references or benchmarks
for evaluating the LLMs responses
Evaluate chat interactions for adherence to
conversational design principles, ensuring responses
are
accurate, relevant, and delivered within acceptable
time frames.
Perform thorough regression testing when bugs are
resolved and when there are updates to LLMs or
application features.
Collaborate with development teams to understand
application features and integration points for
Large Language Models, identifying potential areas
for automation.
Execute automated and manual test cases, analyze
test results, identify and document defects, and
work closely with developers to ensure they are
addressed promptly.
Monitor the performance of LLMs in the chat
interface, identifying any issues with latency,
accuracy,
or relevance of responses.
Stay up-to-date with the latest developments in AI,
machine learning, and chatbot technologies to
continually improve testing strategies and tools.
Collaborate with product managers and UX designers
to ensure the chat application meets the highest
standards of user satisfaction and engagement.
Ensure compliance with data privacy and security
policies during the testing process, especially when
handling sensitive user data.
Provide detailed reports and documentation on test
activities, results, insights, and recommendations
for improving the application& quality and

performance.
Tools Exposure Expected:.

Mandatory skill sets: Python, Machine Learning, Prompt Engineering, Java-

Selenium, Python Robot Framework

Preferred skill sets: LLM, Data Processing, SQL, Azure, CI/CD, Jira

Years of experience required: 4 to 7 years

Education qualification: B.E/B.Tech

Education (if blank, degree and/or field of study not specified)

Certifications (if blank, certifications not specified)

Required Skills

Optional Skills

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Available for Work Visa Sponsorship

Government Clearance Required

Job Posting End Date