Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models.

Alina Shutova, Vladimir Malinovskii, Vage Egiazarian, Denis Kuznedelev, Denis Mazur, Nikita Surkov, Ivan Ermakov, Dan Alistarh

Browse the full ICML paper archive.