Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method.

Shicheng Tan, Weng Lam Tam, Yuanchun Wang, Wenwen Gong, Shu Zhao, Peng Zhang, Jie Tang

Browse the full ACL paper archive.