Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference.

Libo Zhang, Zhaoning Zhang, Xubaizhou, Rui Li, Zhiliang Tian, Songzhu Mei, Dongsheng Li

Browse the full EMNLP paper archive.