Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers.

Krzysztof Choromanski, Shanda Li, Valerii Likhosherstov, Kumar Avinava Dubey, Shengjie Luo, Di He, Yiming Yang, Tams Sarls, Thomas Weingarten, Adrian Weller

Browse the full AISTATS paper archive.