ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning.

Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, Yuxiong He

Browse the full SC paper archive.