Joshua Ainslie
Publication record assembled from the DBLP archive of ranked conferences.
Papers indexed
22
Venues
6
Active years
2020–2025
Best venue rank
A*
Where they publish
Papers
22 indexed papers, newest first.
| Year | Venue | Title | Authors |
|---|---|---|---|
| 2025 | ICLR | Linear Transformer Topological Masking with Graph Random Features. | Isaac Reid, Kumar Avinava Dubey, Deepali Jain, William F. Whitney, Amr Ahmed, Joshua Ainslie, Alex Bewley, Mithun George Jacob, Aranyak Mehta, David Rendleman, Connor Schenck, Richard E. Turner, Ren Wagner, Adrian Weller, Krzysztof Marcin Choromanski |
| 2025 | ICML | Learning the RoPEs: Better 2D and 3D Position Encodings with STRING. | Connor Schenck, Isaac Reid, Mithun George Jacob, Alex Bewley, Joshua Ainslie, David Rendleman, Deepali Jain, Mohit Sharma, Kumar Avinava Dubey, Ayzaan Wahid, Sumeet Singh, Ren Wagner, Tianli Ding, Chuyuan Fu, Arunkumar Byravan, Jake Varley, Alexey A. Gritsenko, Matthias Minderer, Dmitry Kalashnikov, Jonathan Tompson, Vikas Sindhwani, Krzysztof Marcin Choromanski |
| 2024 | ICLR | Functional Interpolation for Relative Positions improves Long Context Transformers. | Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontan, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli |
| 2024 | NAACL | MEMORY-VQ: Compression for Tractable Internet-Scale Memory. | Yury Zemlyanskiy, Michiel de Jong, Luke Vilnis, Santiago Ontan, William W. Cohen, Sumit Sanghai, Joshua Ainslie |
| 2023 | ACL | FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference. | Michiel de Jong, Yury Zemlyanskiy, Joshua Ainslie, Nicholas FitzGerald, Sumit Sanghai, Fei Sha, William W. Cohen |
| 2023 | ACL | FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction. | Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su, Xiang Zhang, Kihyuk Sohn, Nikolay Glushnev, Renshen Wang, Joshua Ainslie, Shangbang Long, Siyang Qin, Yasuhisa Fujii, Nan Hua, Tomas Pfister |
| 2023 | EMNLP | CoLT5: Faster Long-Range Transformers with Conditional Computation. | Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontan, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai |
| 2023 | EMNLP | GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. | Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrn, Sumit Sanghai |
| 2023 | EMNLP | A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding. | Andrea Burns, Krishna Srinivasan, Joshua Ainslie, Geoff Brown, Bryan A. Plummer, Kate Saenko, Jianmo Ni, Mandy Guo |
| 2023 | EMNLP | mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences. | David C. Uthus, Santiago Ontan, Joshua Ainslie, Mandy Guo |
| 2023 | ICLR | Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints. | Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby |
| 2023 | ICML | Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute. | Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald, Joshua Ainslie, Sumit Sanghai, Fei Sha, William W. Cohen |
| 2022 | ACL | FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction. | Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua, Joshua Ainslie, Renshen Wang, Yasuhisa Fujii, Tomas Pfister |
| 2022 | ACL | Making Transformers Solve Compositional Tasks. | Santiago Ontan, Joshua Ainslie, Zachary Fisher, Vaclav Cvicek |
| 2022 | COLING | Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing. | Yury Zemlyanskiy, Michiel de Jong, Joshua Ainslie, Panupong Pasupat, Peter Shaw, Linlu Qiu, Sumit Sanghai, Fei Sha |
| 2022 | EMNLP | Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT. | James Lee-Thorp, Joshua Ainslie |
| 2022 | NAACL | LongT5: Efficient Text-To-Text Transformer for Long Sequences. | Mandy Guo, Joshua Ainslie, David C. Uthus, Santiago Ontan, Jianmo Ni, Yun-Hsuan Sung, Yinfei Yang |
| 2022 | NAACL | FNet: Mixing Tokens with Fourier Transforms. | James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontan |
| 2021 | ACL | RealFormer: Transformer Likes Residual Attention. | Ruining He, Anirudh Ravula, Bhargav Kanagal, Joshua Ainslie |
| 2021 | ACL | Improving Compositional Generalization in Classification Tasks via Structure Annotations. | Juyong Kim, Pradeep Ravikumar, Joshua Ainslie, Santiago Ontan |
| 2021 | NAACL | ReadTwice: Reading Very Large Documents with Memories. | Yury Zemlyanskiy, Joshua Ainslie, Michiel de Jong, Philip Pham, Ilya Eckstein, Fei Sha |
| 2020 | EMNLP | ETC: Encoding Long and Structured Inputs in Transformers. | Joshua Ainslie, Santiago Ontan, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang |