ZeRO: memory optimizations toward training trillion parameter models.

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He

Browse the full SC paper archive.