Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputers.

Siddharth Singh, Prajwal Singhania, Aditya K. Ranjan, John Kirchenbauer, Jonas Geiping, Yuxin Wen, Neel Jain, Abhimanyu Hans, Manli Shu, Aditya Tomar, Tom Goldstein, Abhinav Bhatele

Browse the full SC paper archive.