Deconstructing What Makes a Good Optimizer for Autoregressive Language Models.

Rosie Zhao, Depen Morwani, David Brandfonbrener, Nikhil Vyas, Sham M. Kakade

Browse the full ICLR paper archive.