Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View.

Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma

Browse the full ICLR paper archive.