HelixPipe: Efficient Distributed Training of Long Sequence Transformers with Attention Parallel Pipeline Parallelism.

Geng Zhang, Shenggan Cheng, Xuanlei Zhao, Ziming Liu, Yang You

Browse the full PPoPP paper archive.