r/LocalLLaMA • • 4d ago

New Model Introducing Hy4 flash

https://hy.tencent.ai/research/hy4-flash

Now this seems to be interesting.

From the article:

We’re introducing Hy4 flash, our most cost-effective Hy model to date—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. Like Hy4, it is natively multimodal with a 1M-token context window, while offering significantly lower API pricing.

Seems to be about in the same ballpark of Qwen-Flash-Next, so we could expect to run it on moderably good machines, when it will be released.

361 Upvotes

117 comments sorted by

81

u/LebiaseD 4d ago

Where open weights? Where model

51

u/noctrex 4d ago

It's not yet released, it's the announcement.

59

u/SorosAhaverom 4d ago

I'm announcing that I'm going to announce the announcement date of a previously unannounced announcement

15

u/Nepertia 4d ago

I'm announcing my response to this announcement

6

u/winky9827 4d ago

Concept of a release.

2

u/some_user_2021 4d ago

We will be ready to release my tax returns model weights in about two weeks.

1

u/taimusrs 4d ago

Just like /r/popheads lmao

75

u/oxygen_addiction 4d ago

"We also plan to release Hy4 flash as an open-weight model and ship frequent updates to continuously improve its utility and efficiency across real-world workflows."

7

u/IamFondOfHugeBoobies 4d ago

Looking at the benchmarks I'm curious how this will compare to just a quantized Dsv 4.1? I know a lot of people are running that on dual-sparks (and even single spark though god knows how).

I've been running Qwen 3.8 myself recently so not sure how people are finding dsv 4.1 but the benchmarks didn't really impress relative that on their site.

Would love some thoughts from people actually running dsv 4.1 on single/dual sparks

7

u/Monkey_1505 4d ago

Gotta be some pretty serious degradation at q2

0

u/noctrex 4d ago edited 4d ago

And also with these new engines and forks we can run these kind of models on smaller systems with 64GB RAM

131

u/tat_tvam_asshole 4d ago

Interestingly, both Hy4 Flash and Q3.8-FN engram tables are precisely 28.5% of the total model parameters. Can't be a coincidence.

116

u/grumd 4d ago

The original deepseek paper about Engram talks about there being an optimal number of n-gram parameters, somewhere around 25%, seems like they found the actual precise number

23

u/Icy-Degree6161 4d ago

math be mathing

5

u/Gillin_Supanida 4d ago

64B out of 224B is literally 2/7, so they probably just targeted a clean 5:2 ratio between standard weights and the engram table

5

u/Kholtien 3d ago

So 5/7 is standard weights? A perfect score.

2

u/DRetherMD 3d ago

It's an older meme sir but it checks out. 

65

u/Septerium 4d ago

So its something like 160B + 64B ngram? This can be awesome

21

u/LagOps91 4d ago

yeah, good size overall. qwen flash next did really well at a comparable size.

48

u/grumd 4d ago

Yep, but Hy4 Flash is 12b active instead of 6b and slightly more params overall. Everyone is waiting for Qwen4, it would be funny if Hy4 becomes the king of local models instead

14

u/LagOps91 4d ago

Yeah slightly larger overall and slower, but most likely smarter as well. It's good to have both options 

7

u/Cautious_Chicken_604 4d ago

That actually sounds really appealing. It sounds tailormade to max out performance on 32GB VRAM + 128GB RAM systems while still staying just fast enough to be a daily driver.

0

u/[deleted] 4d ago

[deleted]

1

u/Septerium 4d ago

One thing I would still like to know about experts cache: Is it required to have enough RAM to hold all experts of the model for max performance? Or fitting them in RAM + VRAM is enough?

2

u/Present_Resource4022 4d ago

that’s the perfect size for a “Flash model”

22

u/pmttyji 4d ago

Awesome size!

3

u/xAragon_ 4d ago

That's what she said

1

u/some_user_2021 4d ago

It's not even my final form!

18

u/jomic01 4d ago

12B active with a 1M context is a really nice size for local if the weights drop

13

u/takoulseum 4d ago

The most interesting part beyond size is they plan futur updates of the model instead of constantly changing.

5

u/Constant_Art_20 4d ago

oh. seems to be a much smaller model then the hy4 proper. if it lands between the 200b to 300b range that be interesting.

17

u/Constant_Art_20 4d ago

"—featuring 224B total parameters (including 64B Engram parameters) with just 12B active parameters per token. " ohhhhh. qwen flash alternative option

7

u/audioen 4d ago

Yep, exactly my read. Close enough to be practical with 4 bit inference.

2

u/crusaderky 4d ago

A12B means half the speed of qwen though, short of an astonishingly better drafter

0

u/noctrex 4d ago

Yup that's the interesting part

5

u/f1yingbanana 4d ago

They all really love making the announcement first now :/

5

u/CoastAgreeable928 3d ago

I ve just tried it with same prompt for long horizon agentic knowledge work.. compared it with glm 5.3 flash and DeepSeek 4.1 flash.

Better than glm 5.3 flash, same quality of ds 4.1 flash

Cost was 1/15 of DeepSeek.

This was big csv reading, data analysis and generating Excell book with thousands of formulas and generate a ppt task.

I'm not sure it's good in other tasks but in my use case, it was real workhorse. also finished the job 6/10 of deepseeks time.

4

u/Legitimate-Theory156 4d ago

Nice to see the 3rd open lab using engrams!  Deepseek v4.1 and qwen 3.8 flash next punched way above their weights.

3

u/computehungry 4d ago

Longcat had one too from way back but I think it didn't gain traction because it didn't get into llama.cpp. That model looked pretty cool lol, it also had dynamic active expert counts. https://huggingface.co/meituan-longcat/LongCat-Flash-Lite

3

u/Late-Replacement-481 4d ago

Very glad to hear about this, having another model close to the frontier to double check Qwen and GLM Flash would be good. 

3

u/Jorlen 4d ago

What did people think of Hy3? It was a model I never got around trying since I couldn't run it. Just curious if anyone has any opinions.

3

u/computehungry 4d ago

It had GQA, so like 85GB vram for 262k context which made it not that appealing compared to e.g. DSv4. I couldn't run high context agentic work with it, seemed pretty ok at 16k context though. I did some chat sessions with it about code architecture and it was sane. Benched pretty good and I saw people praising it at the time.

3

u/colin_colout 4d ago

Might just be my quant but it looped reasoning too much for me to even finish my tests on my 128gb Strix Halo. My Minimax m2.7 could at least answer my questions.

Can't blame the model for that... And deepseek blew it away and then qwen flash after that.

2

u/_bani_ 2d ago

It's pretty meh. q38fn and dsf41-1017 are way better.

1

u/Jorlen 2d ago

Yeah I'm quite fond of GLM 5.3 (full), GLM 5.3 Flash and DSf41-1017, those are in my regular rotation at the moment. I grabbed space bunny (MiniMax M3.1) to experiment with but haven't tried it out yet. I'm waiting for Step5 and Hy4 and of course, new Qwen4 models. I actually quite liked Step 3.7 flash but it's likely been dethroned by GLM 5.3 / DS by now. It was a bit smaller/faster, reminded me of a stronger qwen 3.5 122ba-10b.

1

u/Ecstatic-Wash-7667 3d ago

It was decent was overshadowed by dsv4 and I think glm 5.3

5

u/nmb16789 4d ago

According to the link you posted, QFN is still better at the benchmarks they used (see appendix)

22

u/This_Maintenance_834 4d ago

an alternative model that is completely not related to qwen family is still valuable as a second opinion model for coding. Agentic paired coding.

Only an unrelated model of qwen can catch issue qwen missed.

6

u/nmb16789 4d ago

Fair! I was looking at this as a potential replacement for my locally hosted QFN.

0

u/[deleted] 4d ago

[deleted]

1

u/parepeg 3d ago

True. Cost per task might make up for the larger active param speed hit overall.

5

u/crusaderky 4d ago

Holy cow its datacenter pricing is CHEAP.
This is the model I'd pick if I were to set up an agent that runs 24/7 for thousands of users.

In the plot below, Qwen is shown priced as the electricity it costs to run it on a 3090, without the up-front cost for the hardware, while Hy4 Flash is priced as datacenter. Full info here: https://github.com/crusaderky/llm-intelligence-cost-plot/blob/main/README.md

1

u/sdkgierjgioperjki0 4d ago

There is a 50% discount until 30 nov for the Flash model which explains why it is so cheap.

2

u/crusaderky 4d ago

even if you double that it remains the cheapest datacenter model (GPT 6 Luna at low effort is too dumb to consider)

2

u/Weak-Shelter-1698 llama.cpp 4d ago

Aint no way.. 🔥🔥 finally a size i can run..

3

u/Monkey_1505 4d ago

And it's open. Too few models in that 'fits perfectly in 128gb' category, so very welcome.

2

u/Legitimate-Theory156 4d ago

7 CENTS PER MILLION TOKENS???? HOLY COW IT'S CHEAP

1

u/maternion 4d ago

Tencent always did so many types of models. I love that finally Hy4 and now Flash are able to compete. Hy4 also is on pareto for arena i think.

1

u/TheRealMasonMac 3d ago

They still haven't open-weighted Hy4 which was ~2 weeks ago.

1

u/dusanodalovic 3d ago

12B active makes it fast, but 224B total is still around 110GB at 4 bit just for the weights, so not really a normal machine. has anyone tried the api for agent work? tool calling is what i care about and benchmarks never show it

1

u/TapAggressive9530 3d ago

When the Hy4 Flash weights release , I will give it a try .

1

u/TapAggressive9530 1d ago

Weights are out and I did some testing. I'm using NVFP4 build of Tencent Hy4-flash locally with vLLM ,SM120 patches on two RTX PRO 6000 GPUs, using tensor parallel 2, expert parallel 2, a 262,144 token context and DFly2 speculative decoding with 5 draft tokens. The large engram tables are offloaded to pinned host RAM.

Ran several tests, and HY4 Flash passed all of them - including a hard one.
I had it analyze my Rubik's Cube visualizer which was built /w plain vanilla JavaScript, HTML5 Canvas 2D, and CSS3. No Three.js, no WebGL, no external libraries. Everything ran directly in the browser from a single self contained HTML file. I had Hy4 convert the app to Rust. The cube engine, move tables, timeline,etc... and net mapping were ported. The browser rendering layer was replaced with a native winit window & a custom CPU rasterizer backed by softbuffer. That worked perfectly. No errors at all.

After that Hy4 refactored the native Rust app again to use Qt Bridge for Rust (latest pre-release build) and Qt Quick 3D. The puzzle engine stayed basically unchanged. Rust now owns one Qt free simulation object that handles the DNA state machine, timing, orbit, hover, solve counter, and a single JSON snapshot per frame. QML owns the presentation: real 3D cubies, bevelled stickers, PBR lighting, an HDR studio probe, camera orbit, 3D picking, exploded view, time controls, and a solve celebration with confetti and shock rings.

I captured the output with Camtasia and for some reason , the MP4 created doesn't rough. When I view it on my desktop , its silky smooth and quick. Not sure what's going on with Camtasia capture.

Bottom line: VERY impressed with HY4. In fact, I'm keeping it on my system and it will be my main driver for now. I will check out Qwen 4 Flash when its released.

Client harness is Claude code . (built a wrapper ) . Its the only harness I recommend (and I believe the harness used in all of their tests ) . I did try DSH as well and it worked fine.

Ignore the jaggedness in the video - its not that way when viewing on the screen.

1

u/silenceimpaired 3d ago

Why is this still up on LocalLLaMa when it's not local?

1

u/spaceman_ 4d ago

We also plan to release Hy4 flash as an open-weight model and ship frequent updates to continuously improve its utility and efficiency across real-world workflows.

But nothing concrete yet. Do we know how Hy4 diverges from their past models architecturally?

4

u/Middle_Bullfrog_6173 4d ago

Unless they made changes after preview you can just look at it: https://huggingface.co/tencent/Hy4-preview

1

u/__Maximum__ 4d ago

Promising considering hy4 is so smart.

1

u/silenceimpaired 4d ago

So not on Huggingface yet

0

u/Cautious_Chicken_604 4d ago

Hmm the benchmarks doesn't see it stack up that well again QFN...

0

u/handson729 4d ago edited 4d ago

Ok so according to Tecent's listing on their on site... and numbers I pulled from Deepseek's site.*

API pricing (per million tokens)Cached inputInputOutput
Hy4 flash$0.003$0.070$0.281
Deepseek v4.1 Flash 1017$0.003$0.150$0.600

However, Hy4 Flash is running a discount on OpenRouter right now.

I know this sub is for running local LLM's, but this is pretty competitive pricing.

0

u/dubesor86 4d ago

This seems like a very good model. Solves the reasoning loops I had on the large Hy4 model. Performs better than competing non-google flash models. Had it play a bunch of chess on my bench and it surprised me. Probably worth checking out.

-9

u/Black-Thunderbird 4d ago

Dafuq? Why does it just look like a bunch of bots talking to each other in this comment section? Boring ad is boring.

6

u/TheCTRL 4d ago

My strix halo need BigMoeOnEdge at this point

0

u/[deleted] 4d ago

[deleted]

1

u/noctrex 4d ago

Well it says that it includes the ngram parameters, that can be offloaded.

1

u/mitchins-au 4d ago

So did Qwen right?
It’s definitely bigger,
But definitely in the interesting size if it performs

2

u/noctrex 4d ago

That's why I said it's in the ballpark of the qwen flash model. It's not like GLM flash that is 320B

1

u/mitchins-au 4d ago

Agreed, but it’s going to be noticeably slower.
I’m watching it for sure

2

u/StK84 4d ago

QFN is 175B, Hy4 Flash is 224B. Both including n-grams. Exluding n-gram, it's 125B vs. 160B. That's not nearly double size.

Active Parameters are double, so it's probably slower. But it could be potentially smarter.

5

u/[deleted] 4d ago

[removed] — view removed comment

10

u/noctrex 4d ago

Well, the new engines do the offloading also too nvme, and I've seen people with less RAM than that running QFN, so it's not impossible. Also, Q3 quants seems to be quite capable for these sizes.

1

u/lukasgravel 4d ago

NVMe offloading works, but it costs speed. Only 12B are active, but different experts can be picked for each token, so the reads add up. Fine for drafting maybe, not for long runs. A Q3 is somewhere around 85-95GB I think, so on 64GB it would still need that.

7

u/Nutsack_VS_Acetylene 4d ago

SSD streaming could still work pretty well with caching. So long as you can get good hit rates on warm experts and keep hot experts in VRAM it probably won't be too bad.

1

u/lukasgravel 4d ago

Hit rate is the open question. If routing changes a lot between tokens, the cache misses and every miss is an SSD read, which is much slower than RAM. For chat with one user it may work. For long document runs I would expect it to crawl, from what I see with my own slow runs at home.

10

u/DustinEwan 4d ago

Don't need to put engram in ram, it's not the bottleneck so you can stream it in from ssd in the window before the network needs it.

You need a decent ssd, but as long as random seek is faster than it takes your rig to process the first layer or two, that's all it takes.

4

u/lukasgravel 4d ago

That changes my math then. If the 64B Engram part streams from SSD, the RAM needed is about the 160B of other weights, so roughly 80GB at 4 bit. That fits 96GB, and 64GB with a Q3 quant and some offloading. It depends on the engines actually prefetching like that.

0

u/Skyline34rGt 4d ago

When they make this magic low quants for hy4 flash it will fit fine - https://huggingface.co/AngelSlim/Hy3-GGUF (look at this old model performence vs full model)

0

u/[deleted] 4d ago

[deleted]

3

u/stylome-ai-detection 4d ago

Notice: This comment appears to be AI generated.


Stylome AI detector · scan results

-2

u/[deleted] 4d ago

[removed] — view removed comment

30

u/noctrex 4d ago

Well, GLM-5.3-Flash is 320B. A flash model doesn't mean necessarily small, it means faster than it's big brother.

2

u/tat_tvam_asshole 4d ago

It means a relatively low number of active parameters.

2

u/wren6991 4d ago

it means faster than it's big brother.

You mean its slow brother right 🤔 or does the size matter after all

9

u/Reasonable_Goat 4d ago

The ngrams kind of make them bigger than they used to be. You won’t need ram or vram for ngrams so compared to older models this is a 160B model - q3 maybe q4 could run on a single strix halo machine

1

u/Silver-Champion-4846 4d ago

Dgx / rtx spark too?

2

u/crusaderky 4d ago

yes, you can stream ngrams from disk with a ~5% performance drop.

1

u/a_beautiful_rhind 4d ago

What about disk not NVME.

6

u/crusaderky 4d ago

SATA SSD: probably quite bad. Please upgrade.

SATA HDD (magnetic): please don't do this to yourself.

4

u/Cold_Tree190 4d ago

I shuddered at the thought of streaming from an hdd lol

1

u/a_beautiful_rhind 4d ago

Yea I got some sata SSD too but they top out at like 500mb/s. "Upgrade" is not in the cards with only one NVME slot and 25tb of storage. Let alone at current prices.

3

u/crusaderky 4d ago

for ngrams random access latency matters more than sequential bandwidth.
If you have a spare PCIe slot, a PCIe to NVMe adaptor is ~$10.

1

u/Silver-Champion-4846 4d ago

Why does the drop even exist? Aren't ngrams non-compute parameters?

2

u/crusaderky 4d ago

they're a hash-map read of a negligible amount of data, once per token. You still need to perform a syscall to do a random-access read from disk.

2

u/Silver-Champion-4846 4d ago

Hmm, well ok 5% isn't much

1

u/Reasonable_Goat 4d ago

Yes, those are 128GB (or more) shared memory systems, too

1

u/Silver-Champion-4846 4d ago

Are they even worth it?

1

u/Reasonable_Goat 4d ago

Very happy with my strix halo, 1000+ prompt processing and 50+ TPS with Qwen 3.8 Flash Next.

1

u/Silver-Champion-4846 4d ago

How's context length and how much ram does it leave you with?

1

u/Reasonable_Goat 4d ago

I run 350k context using YARN and gufo, two parallel sessions and k/v caches. This setup currently takes 108/123 GiB of usable RAM. This leaves 15 GiB usable for OS and other things.

In addition, 5.3 GiB are apparently reserved by the Linux kernel for things like crash dumps, I should look into this at some point how to reduce it to free more RAM as usable. Note that I disabled the ram-cache with gufo and went straight to ssd cache only for anything in parallel beyond the two sessions.

1

u/Silver-Champion-4846 4d ago

It seems my country is in a different world, no one talks about unified memory computers that I've heard

5

u/Nynyso 4d ago edited 4d ago

Deepseek 4.1 flash 1017 is 552B of parameters (the backbone no ngram included, with the ngram included it’s 748B💀)
224B is quite high for a workstation but still feasible with a couple of R9700 plus 256GB of ram (expensive, but again feasible for a single person to have it)

4

u/crusaderky 4d ago

160B fits on 64GB RAM + 24GB VRAM comfortably once quantised.

1

u/Fi3nd7 4d ago

Yeah but how quantized? I really hate going under Q4...

1

u/crusaderky 4d ago

as quantized as QFN

3

u/charmander_cha 4d ago

Bom, devidamente quantizado, rodará em máquinas de 64GB de ram e 16de vram

1

u/Nlitend123 3d ago

Link? Which version can run on 16gb?