BurstEngine: An efficient distributed framework for training transformers On extremely Long sequences of over 1M tokens.

Ao Sun, Weilin Zhao, Xu Han, Cheng Yang, Zhiyuan Liu, Chuan Shi, Maosong Sun

Browse the full SC paper archive.