Contrastive Distillation on Intermediate Representations for Language Model Compression.

Siqi Sun, Zhe Gan, Yuwei Fang, Yu Cheng, Shuohang Wang, Jingjing Liu

Browse the full EMNLP paper archive.