FMC-LLM: Enabling FPGAs for Efficient Batched Decoding of 70B+ LLMs with a Memory-Centric Streaming Architecture.

Wenheng Ma, Xinhao Yang, Shulin Zeng, Tengxuan Liu, Libo Shen, Hongyi Wang, Shiyao Li, Jiewen Wang, Yuhan Zhang, Hao Guo, Jintao Li, Ziming Zhang, Zhenhua Zhu, Xuefei Ning, Tsung-Yi Ho, Guohao Dai, Yu Wang

Browse the full FPGA paper archive.