Mast: Efficient Training of Mixture-of-Experts Transformers with Task Pipelining and Ordering.

Wenxiang Lin, Xinglin Pan, Shaohuai Shi, Xuan Wang, Bo Li, Xiaowen Chu

Browse the full ICDCS paper archive.