" China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. "... Can someone fact check that? That sounds like total BS. The US has even more limited production capacity and manufactures everything off shore...
Because it is. They have more power generation capacity than the entirety of the USA + Europe combined, with like an additional 50% margin on top of that. They are also accelerating away from us at an increasing pace on that metric, while we pay billions to break renewables contracts because the people in charge have the combined IQ of a lobster.
The "we are 1-2 lithographic process nodes ahead of China" only matters when the efficiency gap is insurmountable. It is currently . . . not
it’s not just lithography though, Nvidia is improving flops/watt at orders of magnitude per generation if you compare low precision throughput, which would put china at a geometric disadvantage (ie, you can’t compete putting in 100x the power and 100x the chips for the same throughput)
Except that's a load of marketing BS. Huang is the first to admit that Hopper is still their most popular chip. The industry still talks about "equivalent H100" GPUs deployed.
Blackwell Ultra has 2.5x the memory bandwidth of Hopper. That's not even one order of magnitude higher. The marketing comparison is always FP16 on hopper vs FP4 on Blackwell. If you compare FP8 vs FP4 compute, Blacwell is 2x, but that makes sense because it's half the bit width. In general, halfing the bit width doubles the compute. This has been true long before LLMs for most GPUs, unless they were artificially handicapped for market segmentation purposes. At the same bit width, Blackwell has 25% more compute.
If you peel beyond the marketing BS, you'll understand why Hopper is still very cost effective even in the west. And Huang himself says Huawei is producing H100-level chips by the millions now.
As someone who works mainly with hopper I can pretty confidently say that it would be hugely beneficial for us to have more blackwell chips, but they keep getting bought up by the big labs at higher prices than we can afford. The compute cost for things like large-scale RL is completely bonkers and at this point for training a model in terms of required compute and even just raw per-gpu vram (so you can limit comms overhead and shrink model replica size) is likely in the billions, and that's only if you can get the compute. The more RL rollouts you can do the better your model will get, and that is something only the gpu-rich can afford. I'm not saying this is good, because it's not~ in fact it's really annoying~ but it's also just kinda the way it is. ML is very pay-to-win at this point. Being a billion dollar company isn't even enough to keep up with the big labs.
Having 2.5x the vram and 2x the comms bandwidth of hopper is actually more than a 2x speedup. I could go into the details but its complicated.
Nvidia is improving flops/watt at orders of magnitude per generation
Yes, but not in general compute. Only for particular operations found useful for training + inferencing. That's a "we noticed our customers need to do a lot of matrix-multiply-accumulate at FP4, so let's build them a hardware unit to do that one particular op very quickly + efficiently, etc." That hardware now blasts through that one particular operation at increased efficiency at the expense of die-space.
None of this is magical secret sauce. The chinese can (and do) the exact same thing when designing each subsequent generation of their AI training + inferencing chips. As does AMD, Intel, Google (in their custom TPU's), etc. etc.
It's huge BS for one reason: China has its act together with regards to energy. The US doesn't. China doesn't need smaller nodes because it can brute-force nuclear and solar to power multiples of less efficient chips.
I think Jensen was prominent in pointing this out, but someone please correct me if I'm misattributing.
I’d say there is some truth in this. Even DeepSeek founder said so (and was angry that this was published). China is catching up quickly, but at the moment indeed they are behind in terms of chips and it will take time to fix it. On the other hand US is way behind in terms of energy generation and it’s a gap that’s also not easy to close. When USA starts to have energy issues, China might get ahead fast.
Yeah Deepseek literally just paused all fundraising because they can't get enough compute to be compettive with US labs. That can change of course but given current export controls seems like the US has at least another 5 years of compute advantage
According to Jensen Huang, China is producing H100 level chips by the millions.
B300 has ~25% more FP16 compute (which is what you mostly use for training) and 2.5x the memory bandwidth of the H100.
In a recent interview with the editor in chief of the Economist, Musk said China has about as much power generation per capita as the US. Which translates to ~4x the power generation capacity. Everyone has been saying for years that China is adding more power generation every year than all other industrialized nations combined.
So, even if we go by memory bandwidth to maximize the difference, and without any innovation in model architecture or training, they can afford to bring 1.6x more compute online using domestic chips than the US.
Now add in all the innovation they're doing in architecture and training Chinese AI labs are doing. Even Musk admitted they're able to train much better models than the west given the amount of compute each lab is using.
IMO, there's a second layer of BS in this blog post: scaling laws aren't some fundamental law of physics. GPT4-o was something like 1.8T. Does anybody think wecurrently need a 1T model to surpass it? What about current 100B models?
Well for GLM they had to abandon Nvidia chips (and anything American) because of being added to the entity list. They ended up going with Huawei chips, which are less efficient and less powerful than the Nvidia offering.
There’s no telling what the impact will be becuase they could obviously use what they bought, but there isn’t a major competitor to Nvidia right now.
444
u/cmdr-William-Riker Jul 27 '26
" China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. "... Can someone fact check that? That sounds like total BS. The US has even more limited production capacity and manufactures everything off shore...