Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases.

Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Zhewei Yao, Yuxiong He

Browse the full ICML paper archive.