SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs.

Shibo Jie, Yehui Tang, Kai Han, Zhi-Hong Deng, Jing Han

Browse the full ICML paper archive.