System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models.

Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Reza Yazdani Aminadabi, Shuaiwen Leon Song, Samyam Rajbhandari, Yuxiong He

Browse the full PODC paper archive.