Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency.

Ruixiao Li, Fahao Chen, Peng Li

Browse the full IJCAI paper archive.