HSDP: Accelerating Large-scale Model Training via Efficient Sharded Data Parallelism.

Yanqi Hao, Zhiquan Lai, Wei Wang, Shengwei Li, Weijie Liu, Keshi Ge, Dongsheng Li

Browse the full ISPA paper archive.