SingularBit: Exploiting Synergy of Singular Value Decomposition and Low-Bit Quantization for Weight and KV Compression in LLM Inference.

Seongyon Hong, Hyundeok Kong, Jungwan Lee, Sangjin Kim, Hoi-Jun Yoo

Browse the full ISCA paper archive.