Scale Efficiently: Insights from Pretraining and Finetuning Transformers.

Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler

Browse the full ICLR paper archive.