Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference.

Nadav Timor, Jonathan Mamou, Daniel Korat, Moshe Berchansky, Oren Pereg, Moshe Wasserblat, Tomer Galanti, Michal Gordon-Kiwkowitz, David Harel

Browse the full ICLR paper archive.