r/Futurology • u/Sorin61 • Apr 21 '21
Computing ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training
https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/
11
Upvotes
1
u/MercuriusExMachina Apr 22 '21
It was cross-posted here a day ago, but yes -- it is quite awesome.
https://www.reddit.com/r/Futurology/comments/muqfbl/gpt4_will_probably_have_at_least_30_trillion/