Forgetting Transformer: Softmax Attention with a Forget Gate.

Zhixuan Lin, Evgenii Nikishin, Xu Owen He, Aaron C. Courville

Browse the full ICLR paper archive.