A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation.

Yuval Ran-Milo, Hila Ofek, Shahar Mendel

Browse the full ACL paper archive.