ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models.

Chao Zeng, Songwei Liu, Yusheng Xie, Hong Liu, Xiaojian Wang, Miao Wei, Shu Yang, Fangmin Chen, Xing Mei

Browse the full AAAI paper archive.