LLM Compressor, Model Quantization and Sparsification techniques and recipes

Existing slides

  1. LLM compressor walkthrough
    https://docs.google.com/presentation/d/16ZbZh4dm_4FZ6drgHwfTSn-KHhkzWNqD9RKnwtWROWU/edit?slide=id.p#slide=id.p

Existing lab resources

  1. Summit lab for LLM compression
    https://rhpds.github.io/showroom-summit2025-lb2959-neural-magic/modules/index.html

Potential Topics to Cover in the Lab

LLM-Compressor

  • Understanding why you SHOULD NOT quantize your own model (and the small number of use cases where you should)

  • Compressing a model using an existing recipe