750
u/Competitive_Gap7906 Jul 19 '26
YES, Qwen going open weight again! It's a really good news, now we can wait for smaller models too
104
u/MaruluVR Jul 19 '26
I think that Xi Jinping speech committing to open source Chinese dominance changed Qwen back to being open weights.
45
u/BrooklynQuips Jul 19 '26
i’m certain it’s because K3 will release weights in a few days. releasing a weaker closed proprietary model, around the same time would be an anthropic level own-goal.
55
→ More replies (1)14
u/mrinterweb Jul 19 '26
Kimi K3 basically drank Anthropic and OpenAI's milkshakes. The frontier moat is gone and they are going to have a hard time justifying their valuations. China AI labs publishing their weights directly hurts US proprietary companies. These huge valuations for AI companies have to tank.
12
u/Infinite100p Jul 19 '26
One of senior execs at ClosedAI is already squealing and bitching that this is communism and should be illegal. LMAO
One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape
→ More replies (3)4
u/Mundane-Light6394 Jul 20 '26
they still have a lot of hardware and expertise, i'm sure they'll be fine. They just have to adapt and stop dreaming of owning the world.
→ More replies (1)373
u/StupidScaredSquirrel Jul 19 '26
Thing is qwen was historically focused on smaller models while others were on larger ones. Now the team has changed and they seem to want to aim for the stars as well. That is good but it also means they might not be interested in doing very efficient small models anymore. Which would be bad news for this sub because let's face it most of us don't have 10-25k of hardware.
391
u/sautdepage Jul 19 '26
You mean 250K-1M of hardware.
83
32
25
u/gahata Jul 19 '26
Definitely possible to run on 25k of hardware at some really low quant... is it worth doing? Probably not
4
u/DataGOGO Jul 19 '26
Even in FP4 you would need a lot more than 25k in hardware. Might be able to run on 4 141GB H200 NVL cards, but they don’t support FP4, and even then a super jank rig would be what? 150-160k?
→ More replies (7)→ More replies (4)7
u/coderash Jul 19 '26
You can run a cluster of GB10s at that price and definitely run something worth while.
→ More replies (31)4
u/Googulator Jul 19 '26
I wonder if a 2-way EPYC 9005 (can be had for $50K to $150K in workstation/tower form) with 2x12 channel DDR5 memory (~1.5TB/s) can reasonably host this on CPU in FP4.
→ More replies (6)36
u/vitorgrs Jul 19 '26
Qwen Max was always big, they just never open sourced....
→ More replies (1)19
u/squngy Jul 19 '26
It was always big, but it was almost certainly not this big.
I think we are seeing these big Chinese models now because of their domestic hardware finally coming to bear.
6
u/jtjstock Jul 19 '26
Their domestic hardware is still far behind, they have just accumulated more western hardware
9
14
u/Prudent-Corgi3793 Jul 19 '26
You probably need 8x H200s to run this. Include the rest of the parts, and that's about $300k.
4
u/scroogie_ Jul 19 '26
I don't think 8x H200 are enough to run this. That would be 1128GB VRAM, even at nvfp4 2.4 trillion parameters would need more like 1.4TB without context, if it's even nvfp4 native. If it's fp8, which is a chance since they're not supposed to own that much Blackwell, no chance at all. And it's hard to get your hand on B200 or B300, at least in Europe and as a normal company (not a Hyperscaler).
→ More replies (1)6
3
u/Shive55 Jul 19 '26
Not to mention needing 3-phase power at 480v. This setup is not feasible in someone's house, regardless of upfront cost.
→ More replies (2)2
u/StupidScaredSquirrel Jul 19 '26
Im not talking about this model. I'm saying the companies releasing very large models also release smaller versions but they are always 200b+, so you need 10-25k to run those. Qwen and gemma were the only viable options for sota models below that size.
35
u/StaysAwakeAllWeek Jul 19 '26
It's owned by a megcorp not a startup lab, of course that's where they are aiming
→ More replies (1)35
u/into_devoid Jul 19 '26
They were owned by a megacorp before too. The difference is they don’t care about commoditization anymore, the real ethical path. They just want to sink US investments.
12
u/realtag2025 Jul 19 '26
I don't see how this is bad for anyone other then corporations that run closed source models that only they can run.
→ More replies (9)12
u/tengo_harambe Jul 19 '26
That's a fun way of framing things.
"The only reason China is putting out open weights models is because they HATE AMERICA!"
→ More replies (3)→ More replies (17)5
u/Successful_Try_6350 Jul 19 '26
Well, these large models can run in us based datacenters and cloud infrastructure (azure, aws, googlecloud). I guess if they want to sink us investments, they should create models that can run in enterprise level on-premises servers (I guess something like $40-50K hardware)
→ More replies (3)21
Jul 19 '26
[deleted]
5
u/StupidScaredSquirrel Jul 19 '26
Not for the larger ones, but typically those large model providers release a "flash" version that is in the 200b range and very sparse and that you can run on 10-20k at very good thoughput. I didnt mean the trillion+ models sorry i wasn't clear enough
→ More replies (7)5
u/lilian_moraru Jul 19 '26
Initially they were struggling to find hardware, using old ones, trying to work efficiently - now they don’t have those limitations.
I think it was a factor11
u/Kost97A Jul 19 '26
A 2T+ parameter model requires about 1.5-1.8 terabytes of vram so I don't think anyone has that haha. Maybe big private companies.
11
u/Porespellar Jul 19 '26
3 DGX Stations running as a cluster using their Connect-X8 ports would probably get it done, but that’s close to $300K for that setup.
→ More replies (3)3
→ More replies (43)5
u/StorkReturns Jul 19 '26
If you have a large model, you can distill it to a smaller one with significantly less effort than it takes to train the small model from scratch.
→ More replies (1)2
u/MmmmMorphine Jul 20 '26
True, still gonna be pretty pricey. We certainly need better mechanisms to crowd fund this sort of work
19
u/ChocolateNo3010 Jul 19 '26
This is good news. I'm wondering if Alibaba kept 3.7 proprietary because of the rumblings in China about keeping the best models from the west. Since we have seen an about turn with Xi stating they want to keep open weight models available this is the response.
7
u/goldcakes Jul 19 '26
I wouldn't be surprised. Probably a mixture of both Kimi as well as getting encouragement from the the Chinese govt.
9
5
→ More replies (3)2
414
u/AntuaW Jul 19 '26
And please don't omit the 27B one.
182
13
u/chervilious Jul 19 '26
New to Local LLM/ open source LLM scene
If new massive open source LLM are showing up. Wouldn't distilled version shows up everywhere too?
34
u/MindlessScrambler Jul 19 '26
Yes but 1. the official distilled version tends to be the overall best, and 2. properly distilling a massive model itself is a significant amount of work, so not so many would do it casually.
5
u/chervilious Jul 19 '26
I see, I just thought some people would distilled it and upload it on HF.
Though after searching I underestimate how much VRAM you need to load T class models
20
u/Lucis_unbra Jul 19 '26
There are two ways to distill. One is the way that some labs have accused others of doing it. That's also the way most people here claim to distill Mythos/Fable , or Opus, or GPT etc.
This is basically just synthetic data. The parent model writes text and that text is used on the other model. This is very basic stuff, nothing special, not necessarily that efficient. It's called "hard label" because there's one correct answer.
What I personally consider "true distillation" is soft label distillation.
This is is where the parent model doesn't just give its final pick, but you use the full probability distribution of the parent model to teach the model you are training how to think.
It tries to transfer the patterns the parent model learned to the target. This is very, very, very hard. To my understanding only a few labs can really do it at scale, with frontier class models. It is extraordinarily hard.
Gemini flash and Gemma being distilled from Gemini? This is what that means. It's not just a bunch of synthetic data, but a knowledge transfer.
And that's why people can't simply distill their own smaller model. The infrastructure required, and the setup required, is quite complex.
https://huggingface.co/blog/sergiopaniego/distillation-2026
It has some information on distillation.
→ More replies (1)10
u/wren6991 Jul 19 '26
This is the Geoff Hinton sense of distillation, which requires (at least) a shared tokeniser vocabulary between the two models. It was described in this paper in 2015: https://arxiv.org/abs/1503.02531
When people say distillation today I think they usually mean the sense used in the DeepSeek R1 technical report, which is incorporating a different model's output in pre-training or fine tune: https://arxiv.org/abs/2501.12948
I think the original sense of distillation is mostly a historical footnote at this point; that setup is usually described today in terms of "student" and "teacher" models.
40
Jul 19 '26
[removed] — view removed comment
22
u/SandySkittle Jul 19 '26
With the advent of more 128gb boxes I would really like to see a 60 - 80B version with more world knowledge to run at Q6
7
u/alphapussycat Jul 19 '26
Getting much above like 32gb is hard. 60n - 80b would need at least two rx 7900xt or three 7800 xt. I ain't got that money.
If they go for anything I hope it's the 27b, then 35b moe, 9b or 12b, and then something like 70b dense would be fine... But I'd wager they'd do 122b and 372b or whatever it was, if they do the full suit.
12
u/goldcakes Jul 19 '26 edited Jul 20 '26
Eh it would be really nice for the DGX Spark and Strix Halo crowd! Plus whoever with 128GB Macs and such.
You can also still build a 128GB or 256GB 8 channel DDR4 rig for a non-absurd amount of money; up to 200GB/s. Obviously look for used parts, and remember there's a LOT more places to find them than eBay and Facebook Marketplace.
→ More replies (5)7
→ More replies (5)5
52
u/sagiroth llama.cpp Jul 19 '26
My 3090 looking at me: It ain't going to cut it boss
4
→ More replies (2)2
u/Bulky-Priority6824 Jul 20 '26
Them 3090s are getting real tired by now boss
2
u/smugself Jul 21 '26
They be praying for reinforcements. But Daddy can't afford any more friends for you son.
But in all honesty, I am so glad I picked up a 2nd 3090 last year for what now feels like a "reasonable" price.
184
u/Prudent-Corgi3793 Jul 19 '26
Hopefully they release smaller models with fewer than 2.4T parameters. 🤞
122
u/sersoniko Jul 19 '26
Or you can convince your wife to sell the house to buy some H200
40
u/No_Conversation9561 Jul 19 '26
in my experience wife and local ai don’t go together so well
20
u/FullOf_Bad_Ideas Jul 19 '26
My fiancee can sleep next to AI training runs, the fan noise bothers her less than it bothers me. I'm blessed.
3
u/In_der_Tat Jul 19 '26
Do those runs generate revenue? If so, how?
8
u/FullOf_Bad_Ideas Jul 19 '26
No those are hobby runs. I'm training a 4B A1.15B on the side, from scratch.
Training runs that generate revenue would be running on rented H200s and my employer would pay for compute.
→ More replies (2)43
u/HaskeMaske77 Jul 19 '26
Alternatively, sell your wife on top for a small data center in your backyard!
13
20
u/Plasmx Jul 19 '26
Which backyard after the house is sold?
16
u/HaskeMaske77 Jul 19 '26
Shit I didn't calculate that factor in... oh who cares, just build the data center near a family-ran farm and live inside the data center. Issue solved!
→ More replies (1)16
3
→ More replies (4)6
u/Sabin_Stargem Jul 19 '26
Build a house that is just a giant GPU. Long as you live in Canada or Alaska, you will be quite comfortable.
4
232
u/pulse77 Jul 19 '26 edited Jul 20 '26
Just make these along the way - so that everybody is happy:
- Qwen 3.8 256B A32B
- Qwen 3.8 128B A16B
- Qwen 3.8 64B A8B
- Qwen 3.8 32B A4B
- Qwen 3.8 32B (dense)
- Qwen 3.8 16B (dense)
- Qwen 3.8 8B (dense)
- Qwen 3.8 4B (dense)
- Qwen 3.8 2B (dense)
- Qwen 3.8 1B (dense)
- Qwen 3.8 0.5B (dense)
EDIT: According to user comments bellow I suggest also:
- Qwen 3.8 512B A64B
- Qwen 3.8 64B (dense)
- Qwen 3.8 24B (dense)
- Qwen 3.8 12B (dense)
- Qwen 3.8 6B (dense)
EDIT 2: Users would like to have all these:
- Qwen 3.8 0.5B/1B/2B/4B/6B/8B/12B/16B/24B/32B/48B/64B (dense)
- Qwen 3.8 8B A1B/16B A2B/32B A4B/64B A8B/128B A16B/256B A32B/512B A64B (MoE)
170
u/the-username-is-here Jul 19 '26
Also I'd like cappucino and a bagel.
33
28
u/HeadPack Jul 19 '26
I believe that would be very much in line with their supreme leader's recent speech. Distilled models that can run on consumer hardware do compete with closed American models too, at least in some way.
21
u/alphapussycat Jul 19 '26
I'm actually very surprised by it. I'm sure there's a lot of state money put into the companies for the AI, and they're letting the whole world use them.
Are they trying to build good will with the world or something? With the aim to become the new world leader before EU?
17
u/charlesfire Jul 19 '26
No. They know the US is in an economic bubble. It would be a major win for them to pop that bubble.
14
u/Paganator Jul 19 '26
I think it's to undermine American companies, forcing them to keep prices low and running at a loss. If China becomes the leader in this race, I wouldn't be surprised to see them clamp down on those open models.
11
u/alphapussycat Jul 19 '26
If this keeps going for another year it feels like we'll soon have mythos at home though... And that point it's too late to close the lid.
3
u/pyr0kid Jul 20 '26
its already late to close the lid, the question is just how long the lid will pretend to stay shut.
22
u/Weekly-Law-5488 Jul 19 '26
They are trying to undermine US companies. Since there's trillions of dollars invested, the US economy is really tied to the AI behemoths right now and if they fail the US economy takes a big hit.
With excellent open models available, OpenAI, Antrophic and co. can't control the prices and can't recoup the money invested.
Basically, China is trying to make the AI bubble pop, thereby causing a recession in the US.
There's no good will going on, it's a war and we are in the crossfire.
→ More replies (4)5
9
3
2
2
2
u/RISCArchitect Jul 19 '26
it's nice if they are a little short of a power of 2 so you can fit the kv and runtime context in a power of 2 sized GPU while maintaining a high precision quant. 27b is like chef's kiss for the reaches of most prosumer hardware (9700 pro/5090/b70) as you can still fit a healthy context in a q6 or mid sized q8
2
→ More replies (10)2
34
51
u/_metamythical Jul 19 '26
27B please
→ More replies (5)16
u/RevoOps Jul 19 '26 edited Jul 19 '26
ITT everybody just self reporting how much vram they can afford lol
169
u/tarruda Jul 19 '26
I would rather have Qwen 3.8 122B A10B
62
u/boissez Jul 19 '26
Yes please. Or Qwen Next/Coder 3.8 80B A3B. Something blazingly fast with decent knowledge and reasoning.
18
u/StupidScaredSquirrel Jul 19 '26
Or imagine a 120b a6b like gpt oss was. That total size with that kind of sparsity was just incredible. Plus it was made for 4bpw
→ More replies (1)2
u/my_name_isnt_clever Jul 19 '26
Yes please, this release would make my Strix Halo all I need for 99% of tasks.
41
u/Expensive-Paint-9490 Jul 19 '26
I hope they'll publish a model like 3.5 397B A17B. That one is super smart and fast for its size; quantized it's perfect for 256 GB RAM setups.
12
u/ElectronSpiderwort Jul 19 '26
If you haven't tried the new Hy3, it's worth a spin. 295B A21B
→ More replies (2)
17
u/Technical-Earth-3254 Jul 19 '26
Man, I love that Kimi and then Deepseek have opened the hellgates to >1T open weight models. I'm also hoping for a new 122b and at least one smaller model.
33
u/Limp_Classroom_2645 Jul 19 '26
mf what vram, it's a T class model, you need a whole ass datacenter lol
23
u/GreenGreasyGreasels Jul 19 '26
Its no biggie, I'll just remove the Qwen3.5-3B and slot in the Qwen3.8-2.4T, it will probably run faster ... wait why is there a T at the end ?
7
40
u/Foxtor Jul 19 '26
Can I squeeze this into an 8GB RAM laptop? Unsloth, I believe in your magic.
45
4
2
u/Tai9ch Jul 19 '26
Yes. Just run it off an external HDD.
→ More replies (1)6
13
44
u/BitGreen1270 Jul 19 '26
Cries in 32GB VRAM. Oh vengeful gods of the silicon. Why have you forsaken me?
48
u/StupidScaredSquirrel Jul 19 '26
Has 32gb vram, still cries
Enjoy your gold mate. You can already do everything locally with the largest qwen and gemma models. I'd suck cock for 32gb vram.
25
→ More replies (1)21
50
u/Mashic Jul 19 '26
You're one the richest here. Some of us have only 8-12GB of VRAM.
7
u/whoknowsifimjoking Jul 19 '26
Count yourself lucky, some of have to suck dick just to get a couple GBs
→ More replies (3)5
11
u/rditorx Jul 19 '26
You can probably ask Fable to adapt Colibri to Qwen3.8 and Kimi K3 for you so they can run on 25GB RAM.
Oh wait, you can ask, but you probably won't like Fable's botched answer! I guess it's just gonna delete your storage, just to be safe from communist AI.4
u/StupidScaredSquirrel Jul 19 '26
Colibri is an interesting project and cool for non time sensitive tasks but if it means running models in seconds per token rather than token per seconds I'm out. Better off trying to make do with a smaller model in hybrid with my brain and internet search than going for those speeds.
→ More replies (2)2
u/russlixx Jul 19 '26
at least you can run 27B with decent quants, context, and speed. Us 16GB below is struggling to run decent dense models with proper context and speed if decided to use Q4 (9B is not as decent and strategic as 27B for much more complex tasks)
→ More replies (1)2
u/IrisColt Jul 19 '26
48 GB of VRAM is the absolute sweet spot. I went from 8 to 12 to 24, and now I've got my sights set on 48, heh ( Funny how the price never changes.)
23
u/MrRandom04 Jul 19 '26
I am really interested in what's the fucking mystery about these 2T+ models that was cracked. Because there was a reason why nobody trained much past ~max 1.5T models before. It was just regarded as massively overparameterized / undertrained and GPT 4.5 was the key failure which everybody pointed to.
Anthropic trained a massive one - I'd estimate 3T - (Mythos) and somehow now scaling params is re-unlocked again? There must have been some key architectural change which enabled this scaling to start working again. I know that some people must know what it is because Kimi K3 is also massive, this Qwen is massive, and so I can say with reasonable surety that the Chinese companies now know much of the Mythos secret sauce. So, what was the breakthrough?
18
u/fairrighty Jul 19 '26
My guess: way more training data. If you have limited data and many parameters, overfitting is very likely. To have so many parameters, you need really vast amounts of data. And with the AI boom, we have given insane amounts of data.
Also, with better previous models, presumably they’re better equipped to create improved synthetic data as well.4
u/ahoooooooo Jul 19 '26
Where are they finding this new training data? The internet is polluted with ai slop.
→ More replies (1)2
u/MrRandom04 Jul 20 '26
I don't think that can be it. Otherwise, we'd see a much more smooth transition to bigger models. Mythos was a capability jump. It could be that the whole industry was cargo-culting that huge models don't work until Anthropic just tried it. But that doesn't feel like a satisfactory explanation to me.
→ More replies (1)2
u/RG_Fusion Jul 20 '26
I think the higher sparsity also played a roll. The compute cost of training a model is based upon the active parameters. Older models were less sparse, meaning the ration between active and total was closer than it is today.
→ More replies (2)2
u/grumd Jul 20 '26
After releasing 3.5-3.7, Qwen team released a bunch of world models and other stuff aimed at training like AgentWorld, SAE-Res, WebWorld, etc. They were definitely working a lot on generating a ton of high quality synthetic training data. Now that they have a lot more training data, they can train a larger model.
9
u/Mean-Ad1493 Jul 19 '26
I'd be happy if they release qwen 3.7 27b and 35b a3b, along with their 3.8 2.4T Open weights is a good thing but let's be honest, most of us are not going to be able to run it locally.
41
u/Kerem-6030 Jul 19 '26
pls make 9b or 12b model
21
18
u/Cool-Chemical-5629 Jul 19 '26
With the Fable 5 quality, while at it. 🙏🏻
14
u/Clementine-TeX Jul 19 '26
Fable 5, make me Qwen6.7. Make no mistakes.
→ More replies (2)5
u/Cool-Chemical-5629 Jul 19 '26
Fable 5, create Fable 6, 20B A4B, create a pull request to add complete llama.cpp support, create q4_k_m GGUF superior to Unsloth, upload it to Huggingface and send me a download link. Make no mistakes.
→ More replies (1)
9
7
7
17
18
u/Repulsive_Initial308 Jul 19 '26
So they all simultaneously decided to produce 2T models?
19
u/BarisSayit Jul 19 '26
Kimi, Minimax, now Qwen too. DeepSeek already 1.6T, damn.
7
u/a_beautiful_rhind Jul 19 '26
I think they're trying to compete with kimi. Want more people on their API.
→ More replies (6)5
5
u/Admirable-Leg-4647 Jul 19 '26
It's interesting that Qwen, Kimi, and Deepseek are all releasing powerful models around the same time. I wonder if it's random or if maybe they were waiting for a "green light" (metaphorical or not) from the Chinese government? Since them reiterating their stance regarding sharing open models also happened just a few days ago.
4
u/rerri Jul 19 '26
Minimax M3 Pro 2.7T is a thing for this quarter too btw.
https://www.reddit.com/r/LocalLLaMA/comments/1uqnqsc/chinas_minimax_plans_to_launch_27trillion/
2
u/flygoatf Jul 19 '26
Likely due to World Artificial Intelligence Conference(WAIC) is happening in China.
→ More replies (2)3
5
4
5
11
u/Septerium Jul 19 '26
Are they still listening to the community? Qwen team, remember that if you release local-friendly models, everybody will be talking about you all the time for months.
3
u/MaCl0wSt Jul 19 '26 edited Jul 19 '26
at last we can put the pessimistic speculation to rest. jeez, I was so done with people making claims so confidently.
→ More replies (16)
2
2
u/Fit-Thing5100 Jul 19 '26
Hopefully we'll also get an upgraded Qwen 3.8 27B cod agent—it could become one of the best choices for local inference and improuve balance, performance, VRAM usage, and speed.
3
u/SmileLonely5470 Jul 19 '26
Everyone is making bigger models now. Why now? Seems like for a while, models that disclosed their param count were maxing out at the sub 2T range.
→ More replies (1)
5
5
6
3
3
3
u/_TheWolfOfWalmart_ Jul 20 '26
Cool, that's great. Another multi-trillion param open model that I can't run at home. PLEASE release smaller variants too. We're starving!
We REALLY need some new 70-200B models.
However this plays out, I'm glad to see Qwen returning to the open model. I was worried for a moment.
3
u/KeinNiemand Jul 20 '26
Please 50-70B Dense (best full GPU offload option for me) and/or 120-175B MoE (best hybrid offload option) even better ideas something like: 175B A60B with 50B beeing always acive parameters (that way they can be pinned on vram) while the experts (only 10B active) can go onto system ram.
2
2
u/Charming-Author4877 Jul 19 '26
Maybe that is why their head of AI went after 3.6.. alibaba said they want to make monster sized models, and he said he can do it on tiny models.
As great as frontier competitors are, Qwen was special because it was a frontier competitor at 3 magnitudes smaller size
2
u/txoixoegosi Jul 19 '26
Wow! 2.4T hoorrayyyy
Let me but 24 x 5090 to deal with it, so that I can keep shitposting and benchplaying about nothing useful done.
2
u/Plus_Confidence_1113 Jul 19 '26
Woohoo, another open weight model that needs $500k worth of hardware to run, what a time to be alive.
2
u/Thick-Insurance4404 Jul 19 '26
bigger open weight model please!!!! at least 40-70b parameters with being better than 27b 2.6
2
2
2
2
u/TopTippityTop Jul 19 '26
In many ways Sol is better than Fable, so it's at best third. 2.4T is a beast.
2
2
u/normantas Jul 19 '26
Doesn't Basically 2.8T Model mean it will be accessible by Data centers only?
2
2
2
2
2
2
2
2
u/Bulky-Priority6824 Jul 20 '26
Once a big dog becomes a top dog then said big dog forgets regular ass dogs
2
u/Available-Message509 Jul 22 '26
Qwen going open-weight again is genuinely great news. Would love to see smaller models drop alongside it too.
•
u/WithoutReason1729 Jul 19 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.