Simple Local Attentions Remain Competitive for Long-Context Tasks.

Wenhan Xiong, Barlas Oguz, Anchit Gupta, Xilun Chen, Diana Liskovich, Omer Levy, Scott Yih, Yashar Mehdad

Browse the full NAACL paper archive.