Deep Optimizer States: Towards Scalable Training of Transformer Models using Interleaved Offloading.

Avinash Maurya, Jie Ye, M. Mustafa Rafique, Franck Cappello, Bogdan Nicolae

Browse the full Middleware paper archive.