Optimizing Distributed LLM Serving through Request Scheduling and Key-Value Cache Sharing.

Hongye Jiang, Mu Wang, Su Yao, Cui Ting, Ziwei Li, Changqiao Xu

Browse the full ICPADS paper archive.