Deep-to-Bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation.

Ankun Wang, Feng Liu, Zhen Huang, Minghao Hu, Dongsheng Li, Yifan Chen, Xinjia Xie

Browse the full KSEM paper archive.