EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse.

Tianyu Guo, Hande Dong, Yichong Leng, Feng Liu, Cheater Lin, Nong Xiao, Xianwei Zhang

Browse the full EuroPar paper archive.