Less is More: Task-aware Layer-wise Distillation for Language Model Compression.

Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, Tuo Zhao

Browse the full ICML paper archive.