Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention.

Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Franois Fleuret

Browse the full ICML paper archive.