Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding.

Mingyu Jin, Kai Mei, Wujiang Xu, Mingjie Sun, Ruixiang Tang, Mengnan Du, Zirui Liu, Yongfeng Zhang

Browse the full ICML paper archive.