PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applications.

Kuntai Du, Bowen Wang, Chen Zhang, Yiming Cheng, Qing Lan, Hejian Sang, Yihua Cheng, Jiayi Yao, Xiaoxuan Liu, Yifan Qiao, Ion Stoica, Junchen Jiang

Browse the full SOSP paper archive.