Di-PS: System-Algorithm Co-Design for Asynchronous and Heterogeneous Cross-cluster LLM Training at Scale.

Shengwei Li, Qiaoling Chen, Zhiquan Lai, Penglong Jiao, Wenwen Qu, Kun Cai, Jiaxing Li, Peng Sun, Xingcheng Zhang, Xiaoge Deng, Dongsheng Li, Kai Lu, Tianwei Zhang

Browse the full NSDI paper archive.