The Marginal Value of Momentum for Small Learning Rate SGD.

Runzhe Wang, Sadhika Malladi, Tianhao Wang, Kaifeng Lyu, Zhiyuan Li

Browse the full ICLR paper archive.