Softpick: No Attention Sink, No Massive Activations with Rectified Softmax.

Zayd Muhammad Kawakibi Zuhri, Erland Hilman Fuadi, Alham Fikri Aji

Browse the full ACL paper archive.