Small Pre-trained Language Models Can be Fine-tuned as Large Models via Over-Parameterization.

Ze-Feng Gao, Kun Zhou, Peiyu Liu, Wayne Xin Zhao, Ji-Rong Wen

Browse the full ACL paper archive.