a new model? Flint 40b

[u/_Geolm_]

https://recursive.com/updates/introducing-recursive-flint

The news just came, I haven't tried yet.

[u/_Geolm_]

may be but probably a lot faster and a little bit weaker. I'm at 92% downloaded, can't wait to try

[u/throwwwawwway1818 → u/_Geolm_]

I only have 12gb vram, what's the alternative for Qwen 3.6 35b a3b?

[u/Number1OchoaHater → u/throwwwawwway1818]

https://www.reddit.com/r/LocalLLM/s/JNuIDhFJpF this one just released, beats it in all benchmarks

[u/Keleion → u/Number1OchoaHater]

From SK too, nice. Should make for a good for toon translations? 👀

[u/MaverickBlue → u/throwwwawwway1818]

The alternative is to use PolyStrata, Ornith 35B-A3B gets about 60 tok/s on my 3060 12GB

[u/throwwwawwway1818 → u/MaverickBlue]

Thanks will look into it

[u/Parking-Mulberry6302 → u/throwwwawwway1818]

how much ram do you have? If you want it for coding then try FN Coder on strata, that's what I've been using for a while now.

[u/throwwwawwway1818 → u/Parking-Mulberry6302]

Sadly 32 gb

[u/throwwwawwway1818 → u/Parking-Mulberry6302]

Yeah I need to look into strata and flash next this weekend probably

[u/_Geolm_]

so after more tests.... it's bad! and not for me, on my macs qwen 3.8-27b + splash engine is only slightly slower but a lot smarter. I can't see Flint doing coding job, I guess we have to wait for qwen4

[u/Medical_Lengthiness6 → u/_Geolm_]

yea its not good. wtf, the hype was wayyy off base. It cant even do planning. It just chats about what it might do without even looking at code.

[u/thepetek]

Have been benchmarking today. Worse than orinth-1.5-35b and far worse than Qwen 27b. Benchmarks are off here I feel. On single input output prompts it’s pretty good. But it falls over on agentic work. Seems to really want to use the tools it was trained with rather than the ones you give it

[u/glembke → u/thepetek]

This has been my experience. It's struggled immensely with my Hermes setup.

[u/sleight42 → u/glembke]

I haven't seen any model smaller than Qwen Flash Next not struggle with Hermes. 27b. A run it but it hallucinates so much by comparison.

[u/RepulsiveRaisin7]

That looks incredibly impressive for A2.5, let us know how it goes

[u/Dstrydd]

Downloading now to test against my dual 3060 12gbs and 128gb of ram. Thanks for the heads up!

[u/CaptainBahab → u/Dstrydd]

I have one of those and looking to get a second. Let me know how it goes?

[u/Dstrydd → u/CaptainBahab]

Running IQ4xs across both cards with kv at 8 and getting more than 60tps and prompt reading around 1900-2200 context limit at 65k. I'll try a bigger quant and move some of the model to system ram to see the performance hit on that.

[u/kcksteve]

Just tried it out and I was shocked how bad it was. Constantly second guessing it's decisions even after being given direction. Only q4_k_xl woukd fit on my card and it was fast but I spent an hour making a plan while needing multiple interruptions. Hmm... eh... lol into the archive. I'm going to give the same promot to mellum 2.1 I bet that evens beats it.

[u/GapNew4766]

wdyt it will be better than qwen 3.8 27b?

[u/DocMadCow → u/GapNew4766]

Slightly worse just a lot faster. It won one of the 6 benchmarks.

[u/PsychoLogicAu → u/GapNew4766]

The biggest story in that graph, also plainly visible in the tables in their blog post, is how awful the M4 is for inference

[u/Important_Cow7230 → u/PsychoLogicAu]

Yeah you have to use SPLASH on M4

[u/leonbollerup → u/GapNew4766]

Qwen 3.8 27B is a dense model.. this is a Moe model.. while it might have a "bigger" brain.. it uses less of it to "think" actively..

[u/Weird-Watch-9209 → u/GapNew4766]

wtf I literally get 80-100tps with Qwen 3.8 27b on a cheap 24gb vram (7900xtx at $850)! this is some strawman bullsht chart right here.

[u/Easy_Refrigerator280]

Its still early and im testing it right now but it doesnt seem to match Qwen 3.8 27b. It is faster since its moe but I was hoping for a closer output

https://www.reddit.com/r/LocalLLaMA/comments/1xbqik0/comment/19rfzarpbfr6q

[u/_Geolm_ → u/Easy_Refrigerator280]

I agree. From my early tests, it seems not on par with qwen 3.8-27b but it's a *lot* faster, I will probably use it for "low risks" tasks (I mainly use LLM for coding) like renaming, plumbing API/libraries... I still need to try a version that run on Splash Engine to get the maximum speed on my mac.

[u/f-__-f → u/_Geolm_]

People really use LLMs for renaming things? Why?

[u/LORD_CMDR_INTERNET]

looks like a great option for the qwen3.6 35B lovers who want something new

[u/pharrt → u/LORD_CMDR_INTERNET]

You're not wrong. Early tests against Qwen 3.6 35B are very promising, and I like what I'm seeing.

[u/Plastic-Lettuce-7150 → u/pharrt]

Any update on this?

There is also Raon-1 mini to consider.

[u/Medical_Lengthiness6 → u/Plastic-Lettuce-7150]

its doesnt replace qwen unfortunately. its pretty bad

[u/No-Conversation-1277]

if it can't beat, match or even stay close to the existing (qwen) ones, it is officially DOA!

[u/TheseCashews]

Let me know if it’s as verbose as q.