Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?

Tokio Kajitsuka, Issei Sato

Browse the full ICLR paper archive.