SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling.

Nikhil Vyas, Depen Morwani, Rosie Zhao, Itai Shapira, David Brandfonbrener, Lucas Janson, Sham M. Kakade

Browse the full ICLR paper archive.