Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention.

Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, Pengfei Zuo

Browse the full USENIX paper archive.