DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads.

Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, Junxian Guo, Shang Yang, Haotian Tang, Yao Fu, Song Han

Browse the full ICLR paper archive.