Manager/Lead - LLM Models (5-10 yrs)

Role : Manager/Lead - LLM Ops

Experience : 5 - 10 years

Location : Bangalore

Role and Responsibilities :

- Define and disseminate LLMOps best practices

- Evaluate and compare different LLMOps tools to incorporate the best practices.

- Collaborate with architects, engineers, data-scientists, and SMEs to design, implement and oversee the infrastructure requirement for the efficient operation of LLM use-cases in production.

- Develop and maintain CI/CD pipelines for prompts deployment and updates.

- Perform Metrics collection, Performance measurement and feedback analysis of LLM use-cases in production systems.

- Monitor and report the cost for LLM use-cases in production systems.

- Identify and resolve any bottlenecks in production LLM use-cases, including Prompt optimization/tuning.

- Ensure the security of LLM operations through comprehensive risk assessments and the implementation of robust security measures.

Required Technical Skills :

- Overall, 5 to 10 years of experience in AI/ML, with at least 1 year in GenAI and LLM operations.

- Proven experience in productionizing the GenAI projects by applying best LLMOps practices.

- Proven experience in LLM monitoring and observability.

- Practical experience working with LLM models and associated techniques, as well as the tools and frameworks used to implement them, like Langfuse (tool), Langchain (framework), RAG (technique).

- Excellent knowledge of prompt writing and prompt management.

- Good python coding skills.

Required non-technical Skills :

- Excellent presentation and communication skills while interacting with both technical and non-technical stakeholders.

- Motivation to learn new things to stay on top of the GenAI trends by keeping pace with industry advancements in LLM technologies and operational methodologies.

- Resilience and the ability to thrive in a dynamic, fast-paced work environment.