Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads.

Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, Tri Dao

Browse the full ICML paper archive.