On the Distribution, Sparsity, and Inference-time Quantization of Attention Values in Transformers.

Tianchu Ji, Shraddhan Jain, Michael Ferdman, Peter A. Milder, H. Andrew Schwartz, Niranjan Balasubramanian

Browse the full ACL paper archive.