Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs.

Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

Browse the full ICLR paper archive.