Democratizing AI: Open-source scalable LLM training on GPU-based supercomputers

Published in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis (Gordon Bell Finalist), 2024

Recommended citation: Siddharth Singh, Prajwal Singhania, Aditya Ranjan, John Kirchenbauer, Jonas Geiping, Yuxin Wen, Neel Jain, Abhimanyu Hans, Manli Shu, Aditya Tomar, Tom Goldstein, Abhinav Bhatele "Democratizing AI: Open-source scalable LLM training on GPU-based supercomputers." In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 1-14). IEEE. https://ieeexplore.ieee.org/abstract/document/10793182