Pre-training Distillation for Large Language Models: A Design Space Exploration.

Hao Peng, Xin Lv, Yushi Bai, Zijun Yao, Jiajie Zhang, Lei Hou, Juanzi Li

Browse the full ACL paper archive.