How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives.

Xinpeng Wang, Leonie Weissweiler, Hinrich Schtze, Barbara Plank

Browse the full ACL paper archive.