ZeRO-Offload: Democratizing Billion-Scale Model Training.

Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He

Browse the full USENIX paper archive.