LLM evaluation with GuideLLM

Existing lab resources

  1. GuideLLM notebook and lab
    https://redhatquickcourses.github.io/genai-vllm/genai-vllm/1/rhoai_deploy/rhoai_query.html

  2. Link to Git repo with code
    https://github.com/RedHatQuickCourses/genai-apps.git

  3. Performance benchmarking with GuideLLM
    https://redhatquickcourses.github.io/genai-vllm/genai-vllm/1/rhoai_deploy/guide_llm.html

  4. evals_workshop
    https://github.com/taylorjordanNC/evals_workshop

Potential Topics to Cover in the Lab

Running Load Tests with GuideLLM

  • Understanding common LLM metrics

  • Configuring --max-num-seqs based on performance tests