Latency-SLO-Aware Memory Offloading for Large Language Model Inference.

Chenxiang Ma, Hanyu Zhao, Zhisheng Ye, Zehua Yang, Tianhao Fu, Jiaxun Han, Jie Zhang, Yingwei Luo, Xiaolin Wang, Zhenlin Wang, Yong Li, Diyu Zhou

Browse the full ICS paper archive.