Mooncake: Trading More Storage for Less Computation - A KVCache-centric Architecture for Serving LLM Chatbot.

Ruoyu Qin, Zheming Li, Weiran He, Jialei Cui, Feng Ren, Mingxing Zhang, Yongwei Wu, Weimin Zheng, Xinran Xu

Browse the full FAST paper archive.