Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm.

Shaoyi Huang, Dongkuan Xu, Ian En-Hsu Yen, Yijue Wang, Sung-En Chang, Bingbing Li, Shiyang Chen, Mimi Xie, Sanguthevar Rajasekaran, Hang Liu, Caiwen Ding

Browse the full ACL paper archive.