Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling.

Wenda Xu, Rujun Han, Zifeng Wang, Long T. Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, Tomas Pfister

Browse the full ICLR paper archive.