NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time.

Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, Hua Wu

Browse the full ACL paper archive.