AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving.

Kaiyu Huang, Hao Wu, Zhubo Shi, Han Zou, Minchen Yu, Qingjiang Shi

Browse the full CLOUD paper archive.