Attention Sinks Are Provably Necessary in Softmax Transformers: Evidence from Trigger-Conditional Tasks.

Yuval Ran-Milo

Browse the full ACL paper archive.