Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation.

Cheng Chen, Yichun Yin, Lifeng Shang, Zhi Wang, Xin Jiang, Xiao Chen, Qun Liu

Browse the full ICANN paper archive.