AmbitionBox
Search
Jobs
- Reviews
- Salaries
- Interview Questions
- About Company
- Benefits
- Jobs
- Office Photos
- Community
Add office photos
All Filters
62 RELX Group Jobs
Senior Service Development Manager - Evaluation
RELX Group
posted 3hr ago
Job Role Insights
Flexible timing
Key skills for the job
Job Description
Senior Service Development Manager - Evaluation
About Our Team
About the Role
The Service Development, Evaluation Manager (AI/LLM Focus) is responsible for optimizing and evaluating service processes, with a specialized focus on services leveraging Generative AI ( GenAI ), Large Language Models (LLMs), and associated AI tools. This role requires expertise in service operations as well as an understanding of AI technologies to ensure high-quality AI-driven service delivery. The manager will collaborate with AI development teams, operations, and customer-facing teams to enhance service efficiency and drive innovation using AI.
Responsibilities
AI/LLM-Based Service Evaluation and Improvement
- Evaluate and assess AI-driven service processes, focusing on the use of Generative AI and LLMs for customer support, content generation, and other service functions.
- Analyze the performance of AI systems and suggest enhancements to improve accuracy, efficiency, and user satisfaction.
- Lead continuous improvement initiatives to optimize service quality, customer satisfaction, and operational efficiency.
- Collaborate with AI development teams to improve model performance, including tuning, prompt engineering, and data refinement for better outputs.
Subject Matter Expertise in Generative AI and LLMs
- Serve as the in-house expert for technical or operational service-related queries on the Generative AI and Large Language Models in service delivery processes.
- Conduct knowledge-sharing sessions and training workshops to enhance service capabilities.
- Stay up-to-date with advancements in AI technologies, including new LLMs, evaluation methods, and AI governance practices.
- Act as a point of contact for escalated service issues, ensuring timely resolution of complex problems.
AI Tool Evaluation and Selection
- Assess various AI evaluation tools and platforms, such as model monitoring systems, accuracy measurement tools, and error detection frameworks.
- Define criteria for evaluating AI model performance, including metrics for language model accuracy, generative quality, and user satisfaction.
- Collaborate with the AI engineering team to integrate evaluation tools that can continuously monitor model output quality and provide insights for further improvement.
- Recommend and implement best practices for using AI safely and ethically, ensuring compliance with industry standards and company policies.
Collaboration with AI and Operations Teams
- Work closely with AI engineers, data scientists, and service operations teams to align service strategies with AI capabilities.
- Provide feedback to teams on refining LLM outputs based on customer needs and service requirements.
- Facilitate cross-functional communication, ensuring that AI-driven service goals are clearly understood and properly implemented.
- Act as the escalation point for AI-related service issues, collaborating with AI teams to resolve complex issues quickly.
Documentation and Reporting of AI-Driven Processes
- Create and maintain documentation for AI/LLM service processes, outlining best practices for using AI tools, handling escalations, and improving outputs.
- Track service performance through key metrics such as response accuracy, speed, user engagement, and overall model effectiveness.
- Generate reports for stakeholders on the performance of AI systems, including analysis of failures, successes, and recommendations for future improvements.
- Maintain a knowledge base on AI/LLM usage within the service department, ensuring up-to-date resources for team members.
Requirements
- Bachelor s degree in a relevant field (e.g., AI, Computer Science, Data Science, or Business with a focus on AI) or equivalent work experience.
- Proven experience as a subject matter expert in service delivery with a focus on AI and LLM-driven services.
- Strong project management skills.
- Knowledge of Generative AI technologies, including hands-on experience with popular Large Language Models (e.g., GPT, LLaMA , PaLM ).
- Familiarity with AI evaluation tools and techniques for assessing model accuracy and quality.
- Strong analytical, problem-solving, and communication skills, with the ability to work cross-functionally.
Preferred Skills
- Certifications in AI/ML technologies.
- Experience with prompt engineering.
- Familiarity with data analysis tools such as Python, R, or LangChain for LLM applications.
Key Competencies
- AI Expertise Deep understanding of AI and LLM technologies, with the ability to translate this knowledge into practical service applications.
- Innovation Ability to identify new ways AI can improve service processes and customer satisfaction.
- Collaboration Strong cross-functional collaboration skills, particularly with AI, data science, and service operations teams.
- Ethics and Governance Awareness of AI ethics, data privacy, and the responsible use of AI in service environments.
- Problem Solving Ability to address complex AI-related service issues quickly and efficiently, ensuring continuous service improvement.
Work in a way that works for you
We promote a healthy work/life balance across the organization. We offer an appealing working prospect for our people. With numerous wellbeing initiatives, shared parental leave, study assistance , and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals.
Working for You
We understand that your well-being and happiness are essential to a successful career. Here are some benefits we offer
- Comprehensive Health Insurance covering you and your immediate family.
- Enhanced Health Insurance Options with competitive rates.
- Group Life Insurance ensuring financial security for your loved ones.
- Group Accident Insurance offering extra protection.
- Flexible Working Arrangements to achieve a balanced work-life schedule.
- Employee Assistance Program providing support for personal and work-related challenges.
- Medical Screening prioritizing your well-being.
- Modern Family Benefits including maternity, paternity, and adoption support.
- Long-Service Awards recognizing dedication and commitment.
- Celebratory New Baby Gift.
- Various Paid Time Off options including Casual Leave, Sick Leave, Privilege Leave, Compassionate Leave, Special Sick Leave, and Gazetted Public Holidays.
About the Business
Employment Type: Full Time, Permanent
Read full job descriptionRELX Group Interview Questions & Tips
Prepare for RELX Group roles with real interview advice
What people at RELX Group are saying
What RELX Group employees are saying about work life
based on 102 employees
Flexible timing
Monday to Friday
No travel
Day Shift
RELX Group Benefits
View more benefits
Compare RELX Group with
Thomson Reuters
Compare
Wolters Kluwer
Compare
Pearson
Compare
Springer Nature in India
Compare
Informa
Compare
Wiley
Compare
Taylor & Francis
Compare
SAGE Publications
Compare
Cengage
Compare
McGraw-Hill Education
Compare
EXL Service
Compare
Access Healthcare
Compare
S&P Global
Compare
AGS Health
Compare
Straive
Compare
Kantar
Compare
Nielsen
Compare
Serco
Compare
Grass Roots Research & Creation
Compare
Coronis Health
Compare
Similar Jobs for you
Share an Interview