QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache.

Rishabh Tiwari, Haocheng Xi, Aditya Tomar, Coleman Richard Charles Hooper, Sehoon Kim, Maxwell Horton, Mahyar Najibi, Michael W. Mahoney, Kurt Keutzer, Amir Gholami

Browse the full ICML paper archive.