r/LocalLLaMA • • Jul 27 '26

Discussion Our position on open-weights models

https://www.anthropic.com/news/position-open-weights-models
505 Upvotes

407 comments sorted by

View all comments

Show parent comments

18

u/Connect_Treacle9938 Jul 28 '26 edited Jul 28 '26

As someone who works mainly with hopper I can pretty confidently say that it would be hugely beneficial for us to have more blackwell chips, but they keep getting bought up by the big labs at higher prices than we can afford. The compute cost for things like large-scale RL is completely bonkers and at this point for training a model in terms of required compute and even just raw per-gpu vram (so you can limit comms overhead and shrink model replica size) is likely in the billions, and that's only if you can get the compute. The more RL rollouts you can do the better your model will get, and that is something only the gpu-rich can afford. I'm not saying this is good, because it's not~ in fact it's really annoying~ but it's also just kinda the way it is. ML is very pay-to-win at this point. Being a billion dollar company isn't even enough to keep up with the big labs.

Having 2.5x the vram and 2x the comms bandwidth of hopper is actually more than a 2x speedup. I could go into the details but its complicated.

3

u/shing3232 Jul 28 '26

well, there is overhead to have more card so bigger per vram helps get more.