Speculative Streaming: Efficient and Scalable Speculative Decoding with Multi-Stream Attention.

Nikhil Bhendawade, Irina Belousova, Qichen Fu, Henry Mason, Antonie Lin, Mohammad Rastegari, Mahyar Najibi

Browse the full EMNLP paper archive.