Libra: Flexible Request Partitioning and Scheduling for Serving Unbalanced and Dynamic LLM Workloads.

Chaoyi Ruan, Yinhe Chen, Dongqi Tian, Yandong Shi, Yongji Wu, Jialin Li, Cheng Li

Browse the full NSDI paper archive.