Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs.

Youhe Jiang, Fangcheng Fu, Xiaozhe Yao, Guoliang He, Xupeng Miao, Ana Klimovic, Bin Cui, Binhang Yuan, Eiko Yoneki

Browse the full ICML paper archive.