The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training.

Jinbo Wang, Mingze Wang, Zhanpeng Zhou, Junchi Yan, Weinan E, Lei Wu

Browse the full ICML paper archive.