SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training.

Chao Ma, Wenbo Gong, Meyer Scetbon, Edward Meeds

Browse the full ICML paper archive.