Yi Tay

Publication record assembled from the DBLP archive of ranked conferences.

Papers indexed

Venues

Active years

2016–2025

Best venue rank

Where they publish

Papers

83 indexed papers, newest first.

Year	Venue	Title	Authors
2025	ACL	BIG-Bench Extra Hard.	Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K. Jain, Virginia Aglietti, Disha Jindal, Peter Chen, Nishanth Dikkala, Gladys Tyen, Xin Liu, Uri Shalit, Silvia Chiappa, Kate Olszewska, Yi Tay, Vinh Q. Tran, Quoc V. Le, Orhan Firat
2024	CVPR	On Scaling Up a Multilingual Vision and Language Model.	Xi Chen, Josip Djolonga, Piotr Padlewski, Basil Mustafa, Soravit Changpinyo, Jialin Wu, Carlos Riquelme Ruiz, Sebastian Goodman, Xiao Wang, Yi Tay, Siamak Shakeri, Mostafa Dehghani, Daniel Salz, Mario Lucic, Michael Tschannen, Arsha Nagrani, Hexiang Hu, Mandar Joshi, Bo Pang, Ceslee Montgomery, Paulina Pietrzyk, Marvin Ritter, A. J. Piergiovanni, Matthias Minderer, Filip Pavetic, Austin Waters, Gang Li, Ibrahim Alabdulmohsin, Lucas Beyer, Julien Amelot, Kenton Lee, Andreas Peter Steiner, Yang Li, Daniel Keysers, Anurag Arnab, Yuanzhong Xu, Keran Rong, Alexander Kolesnikov, Mojtaba Seyedhosseini, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, Radu Soricut
2023	ACL	Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them.	Mirac Suzgun, Nathan Scales, Nathanael Schrli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, Jason Wei
2023	EMNLP	CoLT5: Faster Long-Range Transformers with Conditional Computation.	Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontan, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
2023	EMNLP	DSI++: Updating Transformer Memory with New Documents.	Sanket Vaibhav Mehta, Jai Gupta, Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Jinfeng Rao, Marc Najork, Emma Strubell, Donald Metzler
2023	EMNLP	Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?	Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler
2023	EMNLP	Transcending Scaling Laws with 0.1% Extra Compute.	Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q. Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, Mostafa Dehghani
2023	EMNLP	Symbol tuning improves in-context learning in language models.	Jerry W. Wei, Le Hou, Andrew K. Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc V. Le
2023	EMNLP	Inverse Scaling Can Become U-Shaped.	Jason Wei, Najoung Kim, Yi Tay, Quoc V. Le
2023	ICLR	UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining.	Hyung Won Chung, Xavier Garcia, Adam Roberts, Yi Tay, Orhan Firat, Sharan Narang, Noah Constant
2023	ICLR	Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints.	Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby
2023	ICLR	Language models are multilingual chain-of-thought reasoners.	Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, Jason Wei
2023	ICLR	Recitation-Augmented Language Models.	Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, Denny Zhou
2023	ICLR	UL2: Unifying Language Learning Paradigms.	Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby, Donald Metzler
2023	ICML	Scaling Vision Transformers to 22 Billion Parameters.	Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, Rodolphe Jenatton, Lucas Beyer, Michael Tschannen, Anurag Arnab, Xiao Wang, Carlos Riquelme Ruiz, Matthias Minderer, Joan Puigcerver, Utku Evci, Manoj Kumar, Sjoerd van Steenkiste, Gamaleldin Fathy Elsayed, Aravindh Mahendran, Fisher Yu, Avital Oliver, Fantine Huot, Jasmijn Bastings, Mark Collier, Alexey A. Gritsenko, Vighnesh Birodkar, Cristina Nader Vasconcelos, Yi Tay, Thomas Mensink, Alexander Kolesnikov, Filip Pavetic, Dustin Tran, Thomas Kipf, Mario Lucic, Xiaohua Zhai, Daniel Keysers, Jeremiah J. Harmsen, Neil Houlsby
2023	ICML	The Flan Collection: Designing Data and Methods for Effective Instruction Tuning.	Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, Adam Roberts
2023	SIGIR	Surprise: Result List Truncation via Extreme Value Theory.	Dara Bahri, Che Zheng, Yi Tay, Donald Metzler, Andrew Tomkins
2022	ACL	ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference.	Kai Hui, Honglei Zhuang, Tao Chen, Zhen Qin, Jing Lu, Dara Bahri, Ji Ma, Jai Prakash Gupta, Ccero Nogueira dos Santos, Yi Tay, Donald Metzler
2022	ACL	Sharpness-Aware Minimization Improves Language Model Generalization.	Dara Bahri, Hossein Mobahi, Yi Tay
2022	ACL	Improving Compositional Generalization with Self-Training for Data-to-Text Generation.	Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, Emma Strubell
2022	CVPR	SCENIC: A JAX Library for Computer Vision Research and Beyond.	Mostafa Dehghani, Alexey A. Gritsenko, Anurag Arnab, Matthias Minderer, Yi Tay
2022	EMNLP	Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification.	Jai Gupta, Yi Tay, Chaitanya Kamath, Vinh Tran, Donald Metzler, Shailesh Bavadekar, Mimi Sun, Evgeniy Gabrilovich
2022	ICLR	The Efficiency Misnomer.	Mostafa Dehghani, Yi Tay, Anurag Arnab, Lucas Beyer, Ashish Vaswani
2022	ICLR	ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning.	Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Prakash Gupta, Kai Hui, Sebastian Ruder, Donald Metzler
2022	ICLR	Scarf: Self-Supervised Contrastive Learning using Random Feature Corruption.	Dara Bahri, Heinrich Jiang, Yi Tay, Donald Metzler
2022	ICLR	Scale Efficiently: Insights from Pretraining and Finetuning Transformers.	Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler
2022	ICLR	Charformer: Fast Character Transformers via Gradient-based Subword Tokenization.	Yi Tay, Vinh Q. Tran, Sebastian Ruder, Jai Prakash Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, Donald Metzler
2022	ICML	HyperPrompt: Prompt-based Task-Conditioning of Transformers.	Yun He, Huaixiu Steven Zheng, Yi Tay, Jai Prakash Gupta, Yu Du, Vamsi Aribandi, Zhe Zhao, YaGuang Li, Zhao Chen, Donald Metzler, Heng-Tze Cheng, Ed H. Chi
2022	KDD	A New Generation of Perspective API: Efficient Multilingual Character-level Transformers.	Alyssa Lees, Vinh Q. Tran, Yi Tay, Jeffrey Sorensen, Jai Prakash Gupta, Donald Metzler, Lucy Vasserman
2021	ACL	How Reliable are Model Diagnostics?	Vamsi Aribandi, Yi Tay, Donald Metzler
2021	ACL	StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling.	Yikang Shen, Yi Tay, Che Zheng, Dara Bahri, Donald Metzler, Aaron C. Courville
2021	ACL	Are Pretrained Convolutions Better than Pretrained Transformers?	Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, Donald Metzler
2021	ACL	On Orthogonality Constraints for Transformers.	Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee
2021	EMNLP	Do Transformer Modifications Transfer Across Implementations and Applications?	Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Fvry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel
2021	ICLR	Are Neural Rankers still Outperformed by Gradient Boosted Decision Trees?	Zhen Qin, Le Yan, Honglei Zhuang, Yi Tay, Rama Kumar Pasumarthi, Xuanhui Wang, Michael Bendersky, Marc Najork
2021	ICLR	Long Range Arena : A Benchmark for Efficient Transformers.	Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, Donald Metzler
2021	ICLR	HyperGrid Transformers: Towards A Single Model for Multiple Tasks.	Yi Tay, Zhe Zhao, Dara Bahri, Donald Metzler, Da-Cheng Juan
2021	ICLR	Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters.	Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan, Anh Tuan Luu, Siu Cheung Hui, Jie Fu
2021	ICML	OmniNet: Omnidirectional Representations from Transformers.	Yi Tay, Mostafa Dehghani, Vamsi Aribandi, Jai Prakash Gupta, Philip Pham, Zhen Qin, Dara Bahri, Da-Cheng Juan, Donald Metzler
2021	ICML	Synthesizer: Rethinking Self-Attention for Transformer Models.	Yi Tay, Dara Bahri, Donald Metzler, Da-Cheng Juan, Zhe Zhao, Che Zheng
2021	NAACL	Knowledge Router: Learning Disentangled Representations for Knowledge Graphs.	Shuai Zhang, Xi Rao, Yi Tay, Ce Zhang
2021	WSDM	Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study.	Dara Bahri, Yi Tay, Che Zheng, Cliff Brunk, Donald Metzler, Andrew Tomkins
2020	AAAI	Multi-Level Head-Wise Match and Aggregation in Transformer for Textual Sequence Matching.	Shuohang Wang, Yunshi Lan, Yi Tay, Jing Jiang, Jingjing Liu
2020	ACL	Reverse Engineering Configurations of Neural Text Generation Models.	Yi Tay, Dara Bahri, Che Zheng, Clifford Brunk, Donald Metzler, Andrew Tomkins
2020	ACL	Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences.	Yi Tay, Donovan Ong, Jie Fu, Alvin Chan, Nancy F. Chen, Anh Tuan Luu, Chris Pal
2020	ACL	Interactive Machine Comprehension with Information Seeking Agents.	Xingdi Yuan, Jie Fu, Marc-Alexandre Ct, Yi Tay, Chris Pal, Adam Trischler
2020	CVPR	What It Thinks Is Important Is Important: Robustness Transfers Through Input Gradients.	Alvin Chan, Yi Tay, Yew-Soon Ong
2020	EMNLP	Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder.	Alvin Chan, Yi Tay, Yew-Soon Ong, Aston Zhang
2020	ICLR	Jacobian Adversarially Regularized Networks for Robustness.	Alvin Chan, Yi Tay, Yew-Soon Ong, Jie Fu
2020	ICML	Sparse Sinkhorn Attention.	Yi Tay, Dara Bahri, Liu Yang, Donald Metzler, Da-Cheng Juan
2020	SIGIR	Choppy: Cut Transformer for Ranked List Truncation.	Dara Bahri, Yi Tay, Che Zheng, Donald Metzler, Andrew Tomkins
2020	WSDM	HyperML: A Boosting Metric Learning Approach in Hyperbolic Space for Recommender Systems.	Lucas Vinh Tran, Yi Tay, Shuai Zhang, Gao Cong, Xiaoli Li
2019	AAAI	Holographic Factorization Machines for Recommendation.	Yi Tay, Shuai Zhang, Anh Tuan Luu, Siu Cheung Hui, Lina Yao, Tran Dang Quang Vinh
2019	ACL	Robust Representation Learning of Biomedical Names.	Minh C. Phan, Aixin Sun, Yi Tay
2019	ACL	Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives.	Yi Tay, Shuohang Wang, Anh Tuan Luu, Jie Fu, Minh C. Phan, Xingdi Yuan, Jinfeng Rao, Siu Cheung Hui, Aston Zhang
2019	ACL	Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks.	Yi Tay, Aston Zhang, Anh Tuan Luu, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, Siu Cheung Hui
2019	ACL	Confusionset-guided Pointer Networks for Chinese Spelling Check.	Dingmin Wang, Yi Tay, Li Zhong
2019	EMNLP	Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling.	Jinfeng Rao, Linqing Liu, Yi Tay, Hsiu-Wei Yang, Peng Shi, Jimmy Lin
2019	IJCAI	DeepRec: An Open-source Toolkit for Deep Learning based Recommendation.	Shuai Zhang, Yi Tay, Lina Yao, Bin Wu, Aixin Sun
2019	IJCAI	Quaternion Collaborative Filtering for Recommendation.	Shuai Zhang, Lina Yao, Lucas Vinh Tran, Aston Zhang, Yi Tay
2019	SIGIR	Interact and Decide: Medley of Sub-Attention Networks for Effective Group Recommendation.	Lucas Vinh Tran, Tuan-Anh Nguyen Pham, Yi Tay, Yiding Liu, Gao Cong, Xiaoli Li
2018	AAAI	SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring.	Yi Tay, Minh C. Phan, Luu Anh Tuan, Siu Cheung Hui
2018	AAAI	Cross Temporal Recurrent Networks for Ranking Question Answer Pairs.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2018	AAAI	Learning to Attend via Word-Aspect Associative Fusion for Aspect-Based Sentiment Analysis.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2018	ACL	Reasoning with Sarcasm by Reading In-Between.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui, Jian Su
2018	EMNLP	Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2018	EMNLP	Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2018	EMNLP	Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2018	EMNLP	Attentive Gated Lexicon Reader with Contrastive Contextual Co-Attention for Sentiment Classification.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui, Jian Su
2018	IJCAI	Hermitian Co-Attention Networks for Text Matching in Asymmetrical Domains.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2018	ICWSM	CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2018	KDD	Multi-Pointer Co-Attention Networks for Recommendation.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2018	KDD	Multi-Cast Attention Networks.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2018	WWW	Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2018	WSDM	Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2017	AAAI	Non-Parametric Estimation of Multiple Embeddings for Link Prediction on Dynamic Knowledge Graphs.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui
2017	CIKM	NeuPL: Attention-based Semantic Matching and Pair-Linking for Entity Disambiguation.	Minh C. Phan, Aixin Sun, Yi Tay, Jialong Han, Chenliang Li
2017	CIKM	Dyadic Memory Networks for Aspect-based Sentiment Analysis.	Yi Tay, Luu Anh Tuan, Siu Cheung Hui
2017	CIKM	Multi-Task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs.	Yi Tay, Luu Anh Tuan, Minh C. Phan, Siu Cheung Hui
2017	SIGIR	Cross-Device User Linking: URL, Session, Visiting Time, and Device-log Embedding.	Minh C. Phan, Aixin Sun, Yi Tay
2017	SIGIR	Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture.	Yi Tay, Minh C. Phan, Anh Tuan Luu, Siu Cheung Hui
2017	WSDM	Random Semantic Tensor Ensemble for Scalable Knowledge Graph Link Prediction.	Yi Tay, Anh Tuan Luu, Siu Cheung Hui, Falk Brauer
2016	EMNLP	Learning Term Embeddings for Taxonomic Relation Identification Using Dynamic Weighting Neural Network.	Anh Tuan Luu, Yi Tay, Siu Cheung Hui, See-Kiong Ng