DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference.

Jinwei Yao, Kaiqi Chen, Kexun Zhang, Jiaxuan You, Binhang Yuan, Zeke Wang, Tao Lin

Browse the full ICLR paper archive.