How to set AdamW's weight decay as you scale model and dataset size.

Xi Wang, Laurence Aitchison

Browse the full ICML paper archive.