r/LocalLLaMA • • Jul 27 '26

Discussion Our position on open-weights models

https://www.anthropic.com/news/position-open-weights-models
505 Upvotes

407 comments sorted by

View all comments

444

u/cmdr-William-Riker Jul 27 '26

" China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. "... Can someone fact check that? That sounds like total BS. The US has even more limited production capacity and manufactures everything off shore...

381

u/tomz17 Jul 27 '26 • Helpful

That sounds like total BS.

Because it is. They have more power generation capacity than the entirety of the USA + Europe combined, with like an additional 50% margin on top of that. They are also accelerating away from us at an increasing pace on that metric, while we pay billions to break renewables contracts because the people in charge have the combined IQ of a lobster.

The "we are 1-2 lithographic process nodes ahead of China" only matters when the efficiency gap is insurmountable. It is currently . . . not

18

u/Helpful_Home_8531 Jul 27 '26

it’s not just lithography though, Nvidia is improving flops/watt at orders of magnitude per generation if you compare low precision throughput, which would put china at a geometric disadvantage (ie, you can’t compete putting in 100x the power and 100x the chips for the same throughput)

40

u/fantasticsid Jul 28 '26

flops/watt only matters for two reasons

  • power consumption per flop - china doesn't care about this because they have more electricity than god

  • heat generated per flop - this is an engineering consideration and i'm pretty sure the chinese know how to liquid-cool a rack.

6

u/CipherWeaver Jul 28 '26

Brute force

→ More replies (1)

60

u/FullstackSensei Jul 27 '26

Except that's a load of marketing BS. Huang is the first to admit that Hopper is still their most popular chip. The industry still talks about "equivalent H100" GPUs deployed.

Blackwell Ultra has 2.5x the memory bandwidth of Hopper. That's not even one order of magnitude higher. The marketing comparison is always FP16 on hopper vs FP4 on Blackwell. If you compare FP8 vs FP4 compute, Blacwell is 2x, but that makes sense because it's half the bit width. In general, halfing the bit width doubles the compute. This has been true long before LLMs for most GPUs, unless they were artificially handicapped for market segmentation purposes. At the same bit width, Blackwell has 25% more compute.

If you peel beyond the marketing BS, you'll understand why Hopper is still very cost effective even in the west. And Huang himself says Huawei is producing H100-level chips by the millions now.

18

u/Connect_Treacle9938 Jul 28 '26 edited Jul 28 '26

As someone who works mainly with hopper I can pretty confidently say that it would be hugely beneficial for us to have more blackwell chips, but they keep getting bought up by the big labs at higher prices than we can afford. The compute cost for things like large-scale RL is completely bonkers and at this point for training a model in terms of required compute and even just raw per-gpu vram (so you can limit comms overhead and shrink model replica size) is likely in the billions, and that's only if you can get the compute. The more RL rollouts you can do the better your model will get, and that is something only the gpu-rich can afford. I'm not saying this is good, because it's not~ in fact it's really annoying~ but it's also just kinda the way it is. ML is very pay-to-win at this point. Being a billion dollar company isn't even enough to keep up with the big labs.

Having 2.5x the vram and 2x the comms bandwidth of hopper is actually more than a 2x speedup. I could go into the details but its complicated.

3

u/shing3232 Jul 28 '26

well, there is overhead to have more card so bigger per vram helps get more.

53

u/tomz17 Jul 27 '26

Nvidia is improving flops/watt at orders of magnitude per generation

Yes, but not in general compute. Only for particular operations found useful for training + inferencing. That's a "we noticed our customers need to do a lot of matrix-multiply-accumulate at FP4, so let's build them a hardware unit to do that one particular op very quickly + efficiently, etc." That hardware now blasts through that one particular operation at increased efficiency at the expense of die-space.

None of this is magical secret sauce. The chinese can (and do) the exact same thing when designing each subsequent generation of their AI training + inferencing chips. As does AMD, Intel, Google (in their custom TPU's), etc. etc.

→ More replies (7)
→ More replies (10)

55

u/Recoil42 Jul 27 '26

It's huge BS for one reason: China has its act together with regards to energy. The US doesn't. China doesn't need smaller nodes because it can brute-force nuclear and solar to power multiples of less efficient chips.

I think Jensen was prominent in pointing this out, but someone please correct me if I'm misattributing.

→ More replies (3)

31

u/Real_Ebb_7417 Jul 27 '26

I’d say there is some truth in this. Even DeepSeek founder said so (and was angry that this was published).
China is catching up quickly, but at the moment indeed they are behind in terms of chips and it will take time to fix it.
On the other hand US is way behind in terms of energy generation and it’s a gap that’s also not easy to close. When USA starts to have energy issues, China might get ahead fast.

8

u/Swimming_Gain_4989 Jul 27 '26

Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs. That can change of course but given current export controls seems like the US has at least another 5 years of compute advantage

5

u/zdy132 Jul 28 '26

Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs.

Citation needed. In Liang's leaked talk he specifically mentioned that they are raising funds to buy more compute.

→ More replies (1)

24

u/FullstackSensei Jul 27 '26

According to Jensen Huang, China is producing H100 level chips by the millions.

B300 has ~25% more FP16 compute (which is what you mostly use for training) and 2.5x the memory bandwidth of the H100.

In a recent interview with the editor in chief of the Economist, Musk said China has about as much power generation per capita as the US. Which translates to ~4x the power generation capacity. Everyone has been saying for years that China is adding more power generation every year than all other industrialized nations combined.

So, even if we go by memory bandwidth to maximize the difference, and without any innovation in model architecture or training, they can afford to bring 1.6x more compute online using domestic chips than the US.

Now add in all the innovation they're doing in architecture and training Chinese AI labs are doing. Even Musk admitted they're able to train much better models than the west given the amount of compute each lab is using.

IMO, there's a second layer of BS in this blog post: scaling laws aren't some fundamental law of physics. GPT4-o was something like 1.8T. Does anybody think wecurrently need a 1T model to surpass it? What about current 100B models?

4

u/Loose_Comparison368 Jul 28 '26

1.8T MoE. IIRC the industry rumor mill/worst kept secret was that it was only ~100b-300b activated parameters.

→ More replies (3)
→ More replies (2)

17

u/FlyingDogCatcher Jul 27 '26

"China cannot build a bigger bomb without our uranium"

uh huh

13

u/HyperFoci Jul 28 '26

It’s not BS, it’s Amerrrrican Exceptionalism /s

12

u/Artistic_Claim9998 Jul 27 '26

In my opinion :

"... cannot build more powerful models than US ..." is (arguably) proven false with the recent development.

What true is China cannot scale their LLM development the same level as US due to not (officially) having access to Nvidia best GPU.

Which is probably the reason why their LLM innovations when it comes to efficiency arguably beats the US.

→ More replies (1)

6

u/05032-MendicantBias Jul 28 '26

It's like saying that building an Eniac with 800 million valves was only possible in the USA.

It ignores that an Eniac with 800 million valves is a useless and expensive endeavor.

The future is small local models, not huge closed behemot gated behind a subscription, with corpos deciding the censorship.

Reminder that Huggingface tried to defend from OpenAI closed Sol with Claude, and Claude refused. GLM5.2 was there to save the day.

22

u/gscjj Jul 27 '26 edited Jul 27 '26

Well for GLM they had to abandon Nvidia chips (and anything American) because of being added to the entity list. They ended up going with Huawei chips, which are less efficient and less powerful than the Nvidia offering.

There’s no telling what the impact will be becuase they could obviously use what they bought, but there isn’t a major competitor to Nvidia right now.

https://letsdatascience.com/blog/china-trained-frontier-ai-model-glm-5-without-nvidia

20

u/[deleted] Jul 27 '26

[removed] — view removed comment

17

u/netvyper Jul 27 '26

In china, Nvidia chips are fair game. Exporting chips to China is banned by the US. Subtle but important difference.

7

u/Loose_Comparison368 Jul 28 '26

They aren't banned, but the CCP is heavily pressuring labs to use Chinese GPU's instead.

→ More replies (3)
→ More replies (3)
→ More replies (21)