PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference.

Dongjie Yang, Xiaodong Han, Yan Gao, Yao Hu, Shilin Zhang, Hai Zhao

Browse the full ACL paper archive.