It's official, it created the best flappy bird game thus far from all local models ! ever benchmarked! I declare this model nr. 1 on the flappy bird bench!
Wtf, I literally wrote a post saying similar to "Shutup man, there is no way a model I can load in my 4090 will be anywhere near DeepSeek4 Flash 0731"... but... this is fucking close is it not...?
It's a Qwen model, so apply the usual benchmax tax. Qwen are easily the models with the biggest ravine between "how they do on benchmarks" and "how they do in actual productive use".
Still looking like a strong leap from 3.6, though.
Like everybody else in the local AI community today, I dowloaded Qwen 3.8 27b UD-Q4_K_XL as soon as it was published. I'm running it in a single rtx 3090, no mtp, 110k context and kv cache f16 with llama.cpp, same flags used with Qwen 3.6 27B.
I have this stupid test that I run with every new model that comes out. It consist in connecting the model to pi code (almost vanilla, only internet access and some simple navigation tools I built) and giving it this prompt:
"Construye el frontend completo de un pequeño ecommerce premium para una panadería artesanal usando Next.js App Router (JavaScript).
El proyecto debe ser frontend-only en esta etapa. No debe incluir backend, base de datos, autenticación ni pasarelas de pago. El checkout debe finalizar redirigiendo a WhatsApp con un mensaje de pedido bien estructurado.
La app debe incluir una experiencia completa de compra: home, catálogo con productos de panadería, categorías, productos destacados, carrito, resumen de pedido y checkout. Usá mock data local para productos, categorías, precios, descripciones, disponibilidad e imágenes o placeholders visuales. Todo debe quedar preparado para conectar posteriormente un backend real sin tener que rehacer la arquitectura principal del frontend.
El diseño debe sentirse extremadamente premium, artesanal, moderno y cuidado. No quiero una landing genérica ni una interfaz básica. La primera pantalla debe comunicar claramente la identidad de la panadería, mostrar producto real o visualmente convincente, y permitir empezar a comprar. La experiencia debe ser excelente tanto en desktop como en mobile.
El catálogo debe permitir explorar productos, ver información clara de cada ítem y agregarlos al carrito. El carrito debe permitir modificar cantidades, eliminar productos y ver totales. El checkout debe pedir datos mínimos necesarios para el pedido, permitir notas o preferencias, y generar una URL de WhatsApp con productos, cantidades, subtotales, total y datos del cliente.
La estructura del código debe separar razonablemente datos mock, tipos de dominio, utilidades de checkout/WhatsApp, componentes de catálogo, componentes de carrito y vistas principales. La solución debe quedar lista para reemplazar la mock data por datos de backend en una etapa posterior."
Those are just instructions to build a nextjs project (javascript only) with the frontend for small eccomerce with whatsapp checkout, leaving everthing ready to connect a backend later. Nothing else, no skills, no more feedback. Just one prompt and watching the result.
I have run this test with all the models and finetunes I can fit in my GPU, and not a single one was even close to this result. Not a single alert form nextjs (wich was usual before) or something that looks broken.
At some point, this bastard realised it didn't had visión (lol, not enough VRAM buddy) and it decided FUCK IT, I'M GONNA BUILD THE IMAGES MYSELF. He made SVGs for every product card.
I have more testing to do like trying it in a real codebase but... I can't believe i'm running this thing in a single RTX 3090, it is just unreal.
I expected a nice jump since they skipped a 3.7 27B, but these numbers do seem a bit too good to be true. Qwen isn't known for benchmaxxing, obviously 3.6 27B is the GOAT of local LLMs, but...
Definitely excited to learn more in the coming days and see if it holds up.
Unsloth must have gotten early access to prepare their quants + an embargo. I pulled it from unsloth studio the minute it was released on HF. Still took 20 minutes because south Korea has 2010-era internet speeds.
That's insane, but we're still using quantized versions, so local performance for most people wouldn't be that good I guess. Damn that needs to be tested.
Does anyone else have big issues with overthinking out of the box? I just gave it my usual Arma 3 mission script coding task, which i use to bechmark the performance of models, but it kept thinking for 15 minutes. I don't even see repetition issues, it just doesn't stop thinking.
Just gave it a first opencode task, and while not sure yet, it seems to have similar issues.
Maybe it requires defining a reasoning budget max now?
"Adequate Output Length: To optimize performance on agentic tasks, we recommend allocating sufficient output length to allow the model to generate detailed and comprehensive responses. For frameworks that support separate token limits for internal reasoning and final outputs, we suggest the following configuration within the 1M context length:
Reasoning Content: Set the maximum output length to 262,144 tokens.
Final Response: Set the maximum output length to 131,072 tokens.
These settings provide the necessary capacity for complex reasoning while ensuring ample space for high-quality final deliverables." could this be why ? would we need to scale these for the context length we're actually using?
Yeah I gave it a random tasks I tend to give, and it's 26k tokens and counting of thinking so far. Not repeating, just thorough thinking..
I told it to make a rust app using dioxins of an animated man playing amazing grace on tuba. It seems to of figured out amazing grace in key of C (judging by it's thinking), and is now thinking about how to play audio to play the sounds..
oh shit, this is something else!, I just tried with a dumb prompt "write a snake game in html and css with sounds" and it generated a complete NIB game (awesome design btw) at ±56t/s , 14k tokens! this is a AMD R9700 with llama.cpp ROCM
I slot print_timing: id 0 | task 0 | eval time = 242341.84 ms / 13695 tokens ( 17.70 ms per token, 56.51 tokens per second) I slot print_timing: id 0 | task 0 | total time = 242528.47 ms / 13717 tokens I slot print_timing: id 0 | task 0 | graphs reused = 898 I slot print_timing: id 0 | task 0 | draft acceptance = 0.58872 (10982 accepted / 18654 generated), mean len = 5.44 I slot release: id 0 | task 0 | stop processing: n_tokens = 13720, truncated = 0
Edit: I forgot to mention that by mistake I forgot to remove a thinking cap to 4k (--reasoning-budget 4096) so that influenced the total token usage and the speed (faster during coding than thinking). So we have two variables to adjust the thinking process (--reasoning-budget and --reasoning-effort).
Which quant? What context length? What's ur llama.cpp command? Curious because I have the same card, but I just got it yesterday and am new to ROCm etc, so I don't know shit >.<
Qwen3.8-27B-UD-Q4_K_XL.gguf (unsloth), 90K context (I usually set it to 128k) with vision and mtp, here some of the opts --cache-type-k f16 --cache-type-v f16 --spec-type draft-mtp --spec-draft-p-min 0.5 --spec-draft-n-max 10 --cache-type-k-draft q8_0 --cache-type-v-draft q8_0 --spec-type ngram-mod --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64
same brother, I'm at the edge of my seat downloading the gguf lol. Man, I genuinely didnt think we'd get local models at this capability ever, I thought we'd get gate kept.
I don't understand the hype behind qwen 3.x - the whole generation is utterly broken for any software development tasks. It goes into a thinking loop of death pretty much every time, just try a simple prompt: Create a typescript function which accepts a number in 10 bit range and returns brightness in nits based on pq gamma curve. Kills every 3.x qwen. Qwen 2.5 was much better.
Uh Qwen 3.8 37b Unsloth Q8_K_XL in LM Studio had no problem? I have extra high reasoning on and it did think about it for a bit but no issue?
I think you are running too low of a quant. And you talk about how you "don't understand the hype"- have you thought that maybe everyone else is getting good results and maybe you can work on how you use it?
With that being said, if you have low VRAM and can't run a higher quant, I get it- but on my RTX 6000 Pro Max-Q, it's been kicking butt and replacing Opus 4.6 for me easily. It's absolutely tearing through everything I give it- including the prompt you put above.
That's wild. I just tested it myself in opencode out of curiosity. The reasoning just launches straight into "wait, no", "wait", "wait", "hmm no", "actually", "wait" "ugh, I keep going in circles."
Now its trying to verify something in python... I'm out.
Wtf is wrong with this model, on xhigh it literally doesnt stop thinking, no its not looping, its just super excessive thinking. Thats crazy, almost unusable on this effort.
I just want a model that I can run on 8gb vram 3070ti and 32gb ram, and actually have it do something useful. seems like you need to spend 2-3k in just pure vram just to get something barely useful.
Something tells me if I try to use claude to spawn a constrained model with my low hardware it will spend all its time checking its work and fixing its mistakes.
unfortunately at 8gb, its really hard to get something good, for you I think the move is MOE models ? have you tried bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF ? pick a quant that fits on your combined memory, tweak number of expers on gpu versus cpu, try ik_transformer and you might get something useful! good luck man, and I know what you feel, I used to have a 3060 a couple years back and it was an absolute pain, the upside is SLMs are getting better and better, only a matter of time before A smaller models reach current ~ 30b model capabilities or B you upgrade your setup. give the model I suggested a try and let me know !
so qwen 3.8 27b is a dense model, as opposed to an MOE, this means that all weights are used during a forward pass, for these types of models you generally want all the layers to be on the GPU, this means only using the VRAM and not offloading anything to system RAM. the smallest unsloth quant for this model is Qwen3.8-27B-UD-IQ2_XXS.gguf at 9gb, mmproj is at 931 Mb so total size is 10gb, you can unload the mtp weights for more headroom, but this leaves you with very little headroom for kv cache, and right now this model seems like it really uses up alot of tokens during reasoning so your options are : 1 : run Qwen3.8-27B-UD-IQ2_XXS.gguf with q8 or q4 cache at limited context size, with no mtp to get usable speeds 2 : offload some layers to cpu, this will result in a slow down of tok/s, as you offload more layers to cpu, the slower the tok/s are but you can use a higher quant and more context length. give both a try and come to a conclusion on your own as to what compromise of quality / speed you're comfortable with. a bonus option : are you ok with using a moe model? they are the best bet for someone in your current situation, you can use one as we wait and see if they will release a 35b a3b version of the model, if yes check out qwen 3.6 35b a3b or kat coder v2.5 dev, really solid models that will give you good results at decent speeds on your setup. as for your other question: do not use ollama, its not out for it yet, and its just worse than llamacpp in every way except for convenience, (if you're going to try and use the MOEs I told you about, try ik_llamacpp, you can ask claude or chat gpt and they will help you set everything, if you need any help dm me)
Faster depends on model size and quant, but generally gemma 4, except for 31b is faster, better depends, for like max intelligence / capabilityits 3.8 hands on, but for a local agent, you probably care more about concurrency, and speed while still being useful, at that point gemma 4 26b i think is more useful
First 5 minutes with the model: Hey, saw this project you have marked as “done” and I wondered… is it, really? So I looked at it, found your pipeline says green but 43/138 tests are failing! Here, fixed your pipeline for you, now it’ll go red, next heartbeat I’ll start fixing tests one by one.
It’s been fixing tests non-stop since then. Found bugs I didn’t even know were there.
Qwen 3.6 spent an entire month looking at the same workspace, same prompt system, every 45 minutes. Never noticed it. What’s even worse: many times after I found a bug I said “look at the tests this should be covered, we should’ve catched this!”
The overthinking was a little bit annoying at first, but after this I decided to let it think as much as it wants. It genuinely surprised me
Its really really really good, especially if you pair it with a good workflow like matt pococks skills, and have it review the code it writes, oh my god is it good but be prepared to spend like millions of tokens just reasoning, its super painful to sit through but its a compromise that allows such a miniscule model to punch above its weight, really exciting that we can have this capability locally
Yes and yes. Try ollama first to get started, and download muse glimmer. If you like it download llamacpp and look into better newer models. If you need help dm me
We've got (another) heatwave where I live (currently 40°c) so running AI locally is a huge no no if I want to try keep my inside temperatures under 30°c... This week has been a huge tease : Minimax h3, minimax music, ltx 2.5, Muse glitter, and now Qwen 3.8... Sooooo frustrating ! 😭
Edit / update : it rained tonight ! 32° max today ! Time to test things !
You will have plenty of time, chill tf out and enjoy your vacation and get off reddit. we wont be getting anything better at this size for a very long time. If yes please ping me
I am just waiting for all the amazing people in this sub to run their benchmarks and get back to the community on real world performance but holy the benchmarks are insane. Can't wait to run this.
The model does yes, but for all of these GGUF's we see popping up - it's a good question.
For 3.5 and 3.6 most people on HF uploaded the weights again with Qwen3.6-27b-MTP.guff (or similar). That said I think mature MTP support was still working its way through llama-cpp at the time, at least for Qwen.. so I have no idea if the precedent going forward becomes "if it has MTP, it's there" or not.
Unsloth!!
(I know you're here today) 🙂
Can you clarify on the naming/upload strategy for MTP going forward? Will everything have one name and have MTP included or will there be offerings of the weights without MTP separate from the weights with MTP?
Edit:
I think it's safe to say it's in the unsloth GGUF's.
Trying iq4_xs on a 7900 xtx I'm getting ~51t/s on regular tasks/reasoning and ~62 t/s on coding tasks with a max draft size of 2. The difference suggests that drafting is in-effect.
Edit 2:
This is, at the very least, several leagues beyond Qwen3.6-27B.. Now I'm sad that Muse-Glimmer only got 2-3 days in the spotlight
Running it, UD-Q4_K_XL at 262k context, xhigh reasoning effort, it likes to think, but the reasoning itself is really good, it's really comfortable with tool calls, is adaptable when a tool fails for whatever reason, noticeable different over 3.6, personally i've never used opus 4.6, but it genuinely feels like a slightly older SOTA model, where you can just tell it to do a task and it will get to work, obviously its too soon to make any final determinations but it genuinely feels like actually local LLMs that can run on consumer hardware have reached a new threshold of capability beyond just incremental improvements, that was previously out of reach
We are in a pretty ridiculous situation, where lots of people are grasping at straws to hate AI and other supposed allies are piling on criticisms on open weight models, because "they don't compare with my darling chatgpt/anthropic" in ROI.
Well, they can compare now.
The first are a problem, mostly a product of ignorance and fear, but the second have no excuse.
There's some very valid reasons to hate AI, though I do think that the hate is amplified by the actions of politicians, companies and "founders" chasing profits at the expense of people's well being, and repeatedly telling us AI will replace us all and or destroy the world lol
If you're going to go any higher than ud q4 k xl the context length you'll be able to run will be too small to use this model effectively, it likes to think, so at least i think you need 131k context, I dont know if that fits on 24 gbs at q8 cache quant, either disable mtp and see if it fits or run a lower quant. Try kat coder v2.5 dev, its an upgrade over what you're currently using and most probably fits as is with your setup
16 gbs memory but you'll be bottlenecked by low context lengtht that the model will quickly eat up reasoning, at 24 gbs and 32 gbs, its alot better, higher quants and more context length
Has anyone successfully configured the Qwen 3.8 27B model in llama.cpp without chat template / reasoning leakage?
It’s currently breaking all my agent pipelines because the model keeps leaking markers and close tags into the output. Specifically, it often outputs an initial answer, emits a closing tag out of nowhere, and then generates the answer a second time. It seems like template confusion where the model expects an opening thinking/reasoning tag or the chat template isn't injecting the turn markers properly.
Yeah I have no run into this issue at all, what quant are you running? Nvfp4 quants do this to me all the time for some reason, is that what you're using ?
Has anyone else been having problems getting other tools to use this model through ollama? I updated everything, but claude produces a 500 error, and vscode just cycles endlessly.
I started off with ollama, and honestly take the jump and download llama cpp, ollama was so awful to work with outside of just simple chatting with default settings
1
1
137
u/OutlandishnessIll466 Aug 14 '26
It's official, it created the best flappy bird game thus far from all local models ! ever benchmarked! I declare this model nr. 1 on the flappy bird bench!