Sign Up
Welcome home! Here at WTMX you can make new friends and meet people with the same interests as you. All free and ad-free.
540f7509d933316a96358d791db19e67_1024
Machine Learning and AI
Captcha Challenge

Link: #349167

Machine Learning and AI
shared a link
Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a clus... View More
ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training - Microsoft Research
www.microsoft.com
Since the DeepSpeed optimization library was introduced last year, it has rolled out numerous novel optimizations for training large AI models—improving scale, speed, cost, and usability. As large mod...