Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation.

Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh

Browse the full ICLR paper archive.