The underlying structures of self-attention: symmetry, directionality, and emergent dynamics in Transformer training.

Matteo Saponati, Pascal Sager, Pau Vilimelis Aceituno, Thilo Stadelmann, Benjamin F. Grewe

Browse the full ICML paper archive.