Self-Distillation for Further Pre-training of Transformers.

Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi

Browse the full ICLR paper archive.