No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models.

Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao

Browse the full ICLR paper archive.