Prophet: Fine-grained Load Balancing for Parallel Training of Large-scale MoE Models.

Wei Wang, Zhiquan Lai, Shengwei Li, Weijie Liu, Keshi Ge, Yujie Liu, Ao Shen, Dongsheng Li

Browse the full CLUSTER paper archive.