Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract).

Danilo Dordevic, Vukasin Bozic, Joseph Thommes, Daniele Coppola, Sidak Pal Singh

Browse the full AAAI paper archive.