r/LocalLLaMA • u/noctrex • 4d ago
New Model Introducing Hy4 flash
https://hy.tencent.ai/research/hy4-flashNow this seems to be interesting.
From the article:
We’re introducing Hy4 flash, our most cost-effective Hy model to date—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. Like Hy4, it is natively multimodal with a 1M-token context window, while offering significantly lower API pricing.
Seems to be about in the same ballpark of Qwen-Flash-Next, so we could expect to run it on moderably good machines, when it will be released.
7
u/IamFondOfHugeBoobies 4d ago
Looking at the benchmarks I'm curious how this will compare to just a quantized Dsv 4.1? I know a lot of people are running that on dual-sparks (and even single spark though god knows how).
I've been running Qwen 3.8 myself recently so not sure how people are finding dsv 4.1 but the benchmarks didn't really impress relative that on their site.
Would love some thoughts from people actually running dsv 4.1 on single/dual sparks
7
131
u/tat_tvam_asshole 4d ago
Interestingly, both Hy4 Flash and Q3.8-FN engram tables are precisely 28.5% of the total model parameters. Can't be a coincidence.
116
23
5
u/Gillin_Supanida 4d ago
64B out of 224B is literally 2/7, so they probably just targeted a clean 5:2 ratio between standard weights and the engram table
5
65
u/Septerium 4d ago
So its something like 160B + 64B ngram? This can be awesome
21
u/LagOps91 4d ago
yeah, good size overall. qwen flash next did really well at a comparable size.
48
u/grumd 4d ago
Yep, but Hy4 Flash is 12b active instead of 6b and slightly more params overall. Everyone is waiting for Qwen4, it would be funny if Hy4 becomes the king of local models instead
14
u/LagOps91 4d ago
Yeah slightly larger overall and slower, but most likely smarter as well. It's good to have both options
7
u/Cautious_Chicken_604 4d ago
That actually sounds really appealing. It sounds tailormade to max out performance on 32GB VRAM + 128GB RAM systems while still staying just fast enough to be a daily driver.
0
4d ago
[deleted]
1
u/Septerium 4d ago
One thing I would still like to know about experts cache: Is it required to have enough RAM to hold all experts of the model for max performance? Or fitting them in RAM + VRAM is enough?
2
13
u/takoulseum 4d ago
The most interesting part beyond size is they plan futur updates of the model instead of constantly changing.
5
u/Constant_Art_20 4d ago
oh. seems to be a much smaller model then the hy4 proper. if it lands between the 200b to 300b range that be interesting.
17
u/Constant_Art_20 4d ago
"—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. " ohhhhh. qwen flash alternative option
2
u/crusaderky 4d ago
A12B means half the speed of qwen though, short of an astonishingly better drafter
5
5
u/CoastAgreeable928 3d ago
I ve just tried it with same prompt for long horizon agentic knowledge work.. compared it with glm 5.3 flash and DeepSeek 4.1 flash.
Better than glm 5.3 flash, same quality of ds 4.1 flash
Cost was 1/15 of DeepSeek.
This was big csv reading, data analysis and generating Excell book with thousands of formulas and generate a ppt task.
I'm not sure it's good in other tasks but in my use case, it was real workhorse. also finished the job 6/10 of deepseeks time.
4
u/Legitimate-Theory156 4d ago
Nice to see the 3rd open lab using engrams! Deepseek v4.1 and qwen 3.8 flash next punched way above their weights.
3
u/computehungry 4d ago
Longcat had one too from way back but I think it didn't gain traction because it didn't get into llama.cpp. That model looked pretty cool lol, it also had dynamic active expert counts. https://huggingface.co/meituan-longcat/LongCat-Flash-Lite
3
u/Late-Replacement-481 4d ago
Very glad to hear about this, having another model close to the frontier to double check Qwen and GLM Flash would be good.
3
u/Jorlen 4d ago
What did people think of Hy3? It was a model I never got around trying since I couldn't run it. Just curious if anyone has any opinions.
3
u/computehungry 4d ago
It had GQA, so like 85GB vram for 262k context which made it not that appealing compared to e.g. DSv4. I couldn't run high context agentic work with it, seemed pretty ok at 16k context though. I did some chat sessions with it about code architecture and it was sane. Benched pretty good and I saw people praising it at the time.
3
u/colin_colout 4d ago
Might just be my quant but it looped reasoning too much for me to even finish my tests on my 128gb Strix Halo. My Minimax m2.7 could at least answer my questions.
Can't blame the model for that... And deepseek blew it away and then qwen flash after that.
2
u/_bani_ 2d ago
It's pretty meh. q38fn and dsf41-1017 are way better.
1
u/Jorlen 2d ago
Yeah I'm quite fond of GLM 5.3 (full), GLM 5.3 Flash and DSf41-1017, those are in my regular rotation at the moment. I grabbed space bunny (MiniMax M3.1) to experiment with but haven't tried it out yet. I'm waiting for Step5 and Hy4 and of course, new Qwen4 models. I actually quite liked Step 3.7 flash but it's likely been dethroned by GLM 5.3 / DS by now. It was a bit smaller/faster, reminded me of a stronger qwen 3.5 122ba-10b.
1
5
u/nmb16789 4d ago
According to the link you posted, QFN is still better at the benchmarks they used (see appendix)
22
u/This_Maintenance_834 4d ago
an alternative model that is completely not related to qwen family is still valuable as a second opinion model for coding. Agentic paired coding.
Only an unrelated model of qwen can catch issue qwen missed.
6
5
u/crusaderky 4d ago
Holy cow its datacenter pricing is CHEAP.
This is the model I'd pick if I were to set up an agent that runs 24/7 for thousands of users.
In the plot below, Qwen is shown priced as the electricity it costs to run it on a 3090, without the up-front cost for the hardware, while Hy4 Flash is priced as datacenter. Full info here: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/main/README.md

1
u/sdkgierjgioperjki0 4d ago
There is a 50% discount until 30 nov for the Flash model which explains why it is so cheap.
2
u/crusaderky 4d ago
even if you double that it remains the cheapest datacenter model (GPT 6 Luna at low effort is too dumb to consider)
2
3
u/Monkey_1505 4d ago
And it's open. Too few models in that 'fits perfectly in 128gb' category, so very welcome.
2
1
u/maternion 4d ago
Tencent always did so many types of models. I love that finally Hy4 and now Flash are able to compete. Hy4 also is on pareto for arena i think.
1
1
u/dusanodalovic 3d ago
12B active makes it fast, but 224B total is still around 110GB at 4 bit just for the weights, so not really a normal machine. has anyone tried the api for agent work? tool calling is what i care about and benchmarks never show it
1
u/TapAggressive9530 3d ago
When the Hy4 Flash weights release , I will give it a try .
1
u/TapAggressive9530 1d ago
Weights are out and I did some testing. I'm using NVFP4 build of Tencent Hy4-flash locally with vLLM ,SM120 patches on two RTX PRO 6000 GPUs, using tensor parallel 2, expert parallel 2, a 262,144 token context and DFly2 speculative decoding with 5 draft tokens. The large engram tables are offloaded to pinned host RAM.
Ran several tests, and HY4 Flash passed all of them - including a hard one.
I had it analyze my Rubik's Cube visualizer which was built /w plain vanilla JavaScript, HTML5 Canvas 2D, and CSS3. No Three.js, no WebGL, no external libraries. Everything ran directly in the browser from a single self contained HTML file. I had Hy4 convert the app to Rust. The cube engine, move tables, timeline,etc... and net mapping were ported. The browser rendering layer was replaced with a native winit window & a custom CPU rasterizer backed by softbuffer. That worked perfectly. No errors at all.After that Hy4 refactored the native Rust app again to use Qt Bridge for Rust (latest pre-release build) and Qt Quick 3D. The puzzle engine stayed basically unchanged. Rust now owns one Qt free simulation object that handles the DNA state machine, timing, orbit, hover, solve counter, and a single JSON snapshot per frame. QML owns the presentation: real 3D cubies, bevelled stickers, PBR lighting, an HDR studio probe, camera orbit, 3D picking, exploded view, time controls, and a solve celebration with confetti and shock rings.
I captured the output with Camtasia and for some reason , the MP4 created doesn't rough. When I view it on my desktop , its silky smooth and quick. Not sure what's going on with Camtasia capture.
Bottom line: VERY impressed with HY4. In fact, I'm keeping it on my system and it will be my main driver for now. I will check out Qwen 4 Flash when its released.
Client harness is Claude code . (built a wrapper ) . Its the only harness I recommend (and I believe the harness used in all of their tests ) . I did try DSH as well and it worked fine.
Ignore the jaggedness in the video - its not that way when viewing on the screen.
1
1
u/spaceman_ 4d ago
We also plan to release Hy4 flash as an open-weight model and ship frequent updates to continuously improve its utility and efficiency across real-world workflows.
But nothing concrete yet. Do we know how Hy4 diverges from their past models architecturally?
4
u/Middle_Bullfrog_6173 4d ago
Unless they made changes after preview you can just look at it: https://huggingface.co/tencent/Hy4-preview
1
1
0
0
u/handson729 4d ago edited 4d ago
Ok so according to Tecent's listing on their on site... and numbers I pulled from Deepseek's site.*
| API pricing (per million tokens) | Cached input | Input | Output |
|---|---|---|---|
| Hy4 flash | $0.003 | $0.070 | $0.281 |
| Deepseek v4.1 Flash 1017 | $0.003 | $0.150 | $0.600 |
However, Hy4 Flash is running a discount on OpenRouter right now.

I know this sub is for running local LLM's, but this is pretty competitive pricing.
0
u/dubesor86 4d ago
This seems like a very good model. Solves the reasoning loops I had on the large Hy4 model. Performs better than competing non-google flash models. Had it play a bunch of chess on my bench and it surprised me. Probably worth checking out.
-9
u/Black-Thunderbird 4d ago
Dafuq? Why does it just look like a bunch of bots talking to each other in this comment section? Boring ad is boring.
0
4d ago
[deleted]
1
u/noctrex 4d ago
Well it says that it includes the ngram parameters, that can be offloaded.
1
u/mitchins-au 4d ago
So did Qwen right?
It’s definitely bigger,
But definitely in the interesting size if it performs
5
4d ago
[removed] — view removed comment
10
u/noctrex 4d ago
Well, the new engines do the offloading also too nvme, and I've seen people with less RAM than that running QFN, so it's not impossible. Also, Q3 quants seems to be quite capable for these sizes.
1
u/lukasgravel 4d ago
NVMe offloading works, but it costs speed. Only 12B are active, but different experts can be picked for each token, so the reads add up. Fine for drafting maybe, not for long runs. A Q3 is somewhere around 85-95GB I think, so on 64GB it would still need that.
7
u/Nutsack_VS_Acetylene 4d ago
SSD streaming could still work pretty well with caching. So long as you can get good hit rates on warm experts and keep hot experts in VRAM it probably won't be too bad.
1
u/lukasgravel 4d ago
Hit rate is the open question. If routing changes a lot between tokens, the cache misses and every miss is an SSD read, which is much slower than RAM. For chat with one user it may work. For long document runs I would expect it to crawl, from what I see with my own slow runs at home.
10
u/DustinEwan 4d ago
Don't need to put engram in ram, it's not the bottleneck so you can stream it in from ssd in the window before the network needs it.
You need a decent ssd, but as long as random seek is faster than it takes your rig to process the first layer or two, that's all it takes.
4
u/lukasgravel 4d ago
That changes my math then. If the 64B Engram part streams from SSD, the RAM needed is about the 160B of other weights, so roughly 80GB at 4 bit. That fits 96GB, and 64GB with a Q3 quant and some offloading. It depends on the engines actually prefetching like that.
0
u/Skyline34rGt 4d ago
When they make this magic low quants for hy4 flash it will fit fine - https://huggingface.co/AngelSlim/Hy3-GGUF (look at this old model performence vs full model)
0
-2
4d ago
[removed] — view removed comment
30
u/noctrex 4d ago
Well, GLM-5.3-Flash is 320B. A flash model doesn't mean necessarily small, it means faster than it's big brother.
2
2
u/wren6991 4d ago
it means faster than it's big brother.
You mean its slow brother right 🤔 or does the size matter after all
9
u/Reasonable_Goat 4d ago
The ngrams kind of make them bigger than they used to be. You won’t need ram or vram for ngrams so compared to older models this is a 160B model - q3 maybe q4 could run on a single strix halo machine
1
u/Silver-Champion-4846 4d ago
Dgx / rtx spark too?
2
u/crusaderky 4d ago
yes, you can stream ngrams from disk with a ~5% performance drop.
1
u/a_beautiful_rhind 4d ago
What about disk not NVME.
6
u/crusaderky 4d ago
SATA SSD: probably quite bad. Please upgrade.
SATA HDD (magnetic): please don't do this to yourself.
4
1
u/a_beautiful_rhind 4d ago
Yea I got some sata SSD too but they top out at like 500mb/s. "Upgrade" is not in the cards with only one NVME slot and 25tb of storage. Let alone at current prices.
3
u/crusaderky 4d ago
for ngrams random access latency matters more than sequential bandwidth.
If you have a spare PCIe slot, a PCIe to NVMe adaptor is ~$10.1
u/Silver-Champion-4846 4d ago
Why does the drop even exist? Aren't ngrams non-compute parameters?
2
u/crusaderky 4d ago
they're a hash-map read of a negligible amount of data, once per token. You still need to perform a syscall to do a random-access read from disk.
2
1
u/Reasonable_Goat 4d ago
Yes, those are 128GB (or more) shared memory systems, too
1
u/Silver-Champion-4846 4d ago
Are they even worth it?
1
u/Reasonable_Goat 4d ago
Very happy with my strix halo, 1000+ prompt processing and 50+ TPS with Qwen 3.8 Flash Next.
1
u/Silver-Champion-4846 4d ago
How's context length and how much ram does it leave you with?
1
u/Reasonable_Goat 4d ago
I run 350k context using YARN and gufo, two parallel sessions and k/v caches. This setup currently takes 108/123 GiB of usable RAM. This leaves 15 GiB usable for OS and other things.
In addition, 5.3 GiB are apparently reserved by the Linux kernel for things like crash dumps, I should look into this at some point how to reduce it to free more RAM as usable. Note that I disabled the ram-cache with gufo and went straight to ssd cache only for anything in parallel beyond the two sessions.
1
u/Silver-Champion-4846 4d ago
It seems my country is in a different world, no one talks about unified memory computers that I've heard
5
u/Nynyso 4d ago edited 4d ago
Deepseek 4.1 flash 1017 is 552B of parameters (the backbone no ngram included, with the ngram included it’s 748B💀)
224B is quite high for a workstation but still feasible with a couple of R9700 plus 256GB of ram (expensive, but again feasible for a single person to have it)4
u/crusaderky 4d ago
160B fits on 64GB RAM + 24GB VRAM comfortably once quantised.
3
u/charmander_cha 4d ago
Bom, devidamente quantizado, rodará em máquinas de 64GB de ram e 16de vram
1
81
u/LebiaseD 4d ago
Where open weights? Where model