Llumnix: Dynamic Scheduling for Large Language Model Serving.

Biao Sun, Ziming Huang, Hanyu Zhao, Wencong Xiao, Xinyi Zhang, Yong Li, Wei Lin

Browse the full OSDI paper archive.