FoldMoE: Efficient Long Sequence MoE Training via Attention-MoE Pipelining.

Guichao Zhu, Lintian Lei, Yuhao Qing, Yichao Fu, Fanxin Li, Dong Huang, Zekai Sun, Heming Cui

Browse the full ACL paper archive.