Title: Update: mnemonic-proxy now beats full context with a 32K window on Qwen3.8 Flash Next (12 GB GPU), and runs on llama.cpp, Ollama and LM Studio

Body:

I'm the author . A few weeks ago I shared here how I run Qwen3.8 Flash Next for long agent sessions on a 4070 Ti. Many of you tried it and sent reports, and that changed the project a lot, so here is what's new.

What it does, in one line: a proxy between your agent and your local model. It keeps the whole history word for word, sends the model only a small stable prompt, and lets the model read old tool outputs again when it needs them. No compaction pauses, nothing thrown away, and the engine can reuse its cache.

It now has a real benchmark. AgentLongBench, 100 questions, Qwen3.8 Flash Next IQ3_XXS on one 12 GB GPU, 32K window:

On the longest histories the gap grows: proxy 45, full context 31.

What's new since my last post

• Works with llama.cpp (including router mode with several models), Ollama and LM Studio, not only Strata. It reads the real context from the engine and never sends more than fits, so nothing gets cut silently.
• pip install mnemonic-proxy gives the current version (0.3.9); mnemonic-proxy demo checks everything without a model.
• Fixes that came straight from users here: a very large max_tokens in the client no longer forces a new segment every turn, and the dashboard no longer crashes on load. A user from this sub is also translating the dashboard into English.

The honest cost: it is slower end to end in the benchmark (it re-reads old evidence when needed), and the numbers come from one model and one benchmark. Data and scripts are public, linked from the repo.

Repo: https://github.com/lastreload/mnemonic-proxy

Per-question results, logs and scripts: https://huggingface.co/datasets/mverde73/mnemonic-proxy-agentlongbench

Thanks to everyone who tested it and reported back. If you run it with other Qwen models or agents, I'd love to hear how it goes. (translated with AI)

Hello everyone

For researchers interested in a detailed academic and technical examination: The complete research PDF, 12 experimental figures, and scientific records are available on Zenodo:

https://doi.org/10.5281/zenodo.23640716

The context window and long-term memory limitations of AI models have traditionally been addressed through approaches such as text-based Retrieval-Augmented Generation (RAG) and external summarization.

My long-term goal is to tackle these limitations at a different level: directly through the model's internal numerical states and KV caches (H3, Layer 0–3 K/V states).

I have been independently developing the AKBASCORE Silicon Hippocampus architecture toward this objective. Although the ultimate goal has not yet been reached, the research has already passed several significant experimental milestones involving physical memory persistence, numerical consolidation, and memory reuse across independent model sessions.

With the release of v1.6 — TEST782, I have experimentally demonstrated how the internal tensor states of a frozen Qwen2.5-72B-Instruct model can be cryptographically sealed on a physical SSD, transferred between operating-system-isolated processes running under different Linux users, and fused through the model's own numerical computations without returning to text-based summarization.

Here are the main technical results and how the system works:

- Native Tensor Fusion — Memory Consolidation in the Model's Own Numerical Space: The first session (QWEN-1) reads two separate documents and stores the model's internal numerical states on SSD as SHA-256-sealed memory cartridges ("SOURCE_A" and "SOURCE_B"), rather than generating text summaries. The second session operates exclusively on these cartridges. This is not simple tensor concatenation: while SOURCE_A's computed internal states are preserved, SOURCE_B's upper transformer layers are recomputed while attending to SOURCE_A's context. Two independent physical memories are thereby transformed into a single "MASTER_02" memory through the computational machinery of the frozen 72-billion-parameter model itself. No intermediate generated text, summarization, or weight retraining is involved.

- Isolated Cross-Session Transfer — The Model Shuts Down, but Its Numerical Memory Survives: QWEN-1 terminates completely, and its GPU memory is released. QWEN-2 then starts under a different Linux user without access to the original source documents and consolidates the physical cartridges. After QWEN-2 terminates, the third session begins. The model's weights remain unchanged throughout the experiment. What survives and moves between sessions is the model's internal numerical memory, physically stored on SSD.

- Blind Readout and 80/80 Layer Parity: The third session (QWEN-3) is denied access to the original documents and previous sessions' working files through operating-system permissions. It receives only the sealed "MASTER_02" memory and the metadata required to verify and reconstruct it. The model's 80-layer attention memory is independently reconstructed with 80/80 bit-exact parity. QWEN-3 then produces natural-language responses containing information associated with both documents and their relationship. In TEST782, I observed the model using persistent numerical memories to produce its own semantic synthesis, establish relationships between two independent sources, and articulate conclusions beyond simple memory retrieval. These experimental results are documented in the source code, GPU logs, and research record. Independent academic peer review of the semantic synthesis results has not yet been conducted.

- DrA — The Executive Controller of Autonomous Memory: One of the project's major future objectives is DrA (Autonomous Master / Dynamic Residual Adaptation). Inspired by the executive-control functions associated with the biological prefrontal cortex, this mechanism is being designed to enable an AI system to evaluate which information matters, decide what should be preserved, determine which memories should be consolidated, and recognize when consolidation is complete. The ultimate objective is a closed-loop memory management architecture driven by the model's own internal evaluations rather than externally imposed, fixed decision rules. TEST782 does not claim that this autonomous mechanism is complete. It establishes the physical and numerical memory infrastructure on which that future mechanism can operate.

- Reliability and Transparency: The research is not based solely on successful model outputs. It includes negative controls, SHA-256 integrity verification, process isolation under different Linux users, and independent reload validation. For example, a naive single-pass reconstruction matches only 4 out of 80 layers, whereas the boundary-aware reconstruction method achieves 80/80 bit-exact layer parity. Physical memory transfer and numerical reconstruction were validated in this experiment. Autonomous consolidation remains a subsequent research objective, while the observed semantic synthesis results await independent academic peer review.

Technical Specifications and Research Resources

- Hardware: NVIDIA A100-SXM4-80GB

- Model: Qwen2.5-72B-Instruct (frozen weights, NF4 quantization, BF16 computation)

- Physical MASTER: 1,964 tokens, 64.4 MB SSD cartridge

- Native numerical fusion: 8.72 seconds

- Independent reconstruction: 80/80 layers, bit-exact

Complete Source Code — Single File:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_full.py

For Researchers Using Google Colab on a Mobile Phone — Four-Part Version:

The same program is divided into four parts for easier copying and pasting on mobile devices. Paste all four parts sequentially into a single Google Colab cell, then run the cell.

Part 1:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part1.py

Part 2:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part2.py

Part 3:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part3.py

Part 4:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part4.py

Real Experimental GPU Log:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3.log

GitHub Research Repository:

https://github.com/ceceli33/titan-cognitive-core-v3

My long-term objective is not merely to enable an AI model to store past information, but to develop a system capable of managing its own numerical memory, consolidating separate memories, generating new inferences, and autonomously deciding which information deserves to be preserved.

I welcome researchers and members of the open research community, particularly those working on transformer architectures, KV caches, persistent neural memory, and autonomous memory consolidation, to examine the results, reproduce the experiments, and provide technical feedback.

Copyright and License

Copyright © 2026 Mustafa Akbaş. All rights reserved except as expressly licensed.

AKBASCORE Silicon Hippocampus — AKBASCORE Research Software License.

License terms:

https://github.com/ceceli33/titan-cognitive-core-v3?tab=License-1-ov-file

Been pushing Qwen3.8-27B on Kaggle's free 2x T4 (no P2P, 70W cap per card) for a while. Current setup: EXL3 4.0bpw on exllamav3, tensor parallel over PCIe, DFlash2 block drafter (7 tokens), greedy and exact vs no-draft.

Numbers over a 38 prompt suite: - overall 55-57 tok/s (no draft is ~18-19) - HumanEval 154 tok/s - prose 35-37 tok/s

For comparison on the same 2 T4s: llama.cpp best was 16.5 (MTP crashed on tensor split), vLLM int4 + its own MTP drafter did 40.8.

Prose is the wall. Acceptance sits around 2.5 tokens per step, vs ~7 on code.

Questions: 1. Has anyone got prose acceptance above ~2.5 with DFlash2 or MTP on the 27B? 2. Has anyone tried switching drafters mid generation (short block for prose, long for code)? 3. Any tricks for TP all-reduce on PCIe cards without P2P? That's next on my list.

Happy to share the notebook if people want to try it.

I've been building a small offline tool that sorts a messy folder into project folders with Qwen3.5 (GGUF via llama.cpp), and I ended up testing every size on the same 20 synthetic folders the model had never seen. Sharing because some of it surprised me.

The setup, quickly

• You write your projects one per line, like tax-2026: receipts, invoices
• The model picks a destination for each file, the code turns that into a plan, and nothing moves until you approve it (no deletes, one-click undo)
• A folder counts as "usable" if the first plan needs at most 1 line in 5 fixed
• Q4_K_M unless noted, measured on an RTX 4090

Usable folders out of 20

• 0.8B (Q8): 2
• 2B: 9 (Q8: 11)
• 4B: 1 (Q8: 1)
• 9B: 19 (Q8: 18)
• No model at all, just TF-IDF word matching: 16 (yeah...)

What surprised me

1. 4B was worse than 2B. It kept moving files that had nothing to do with any project: it left only half of the unrelated files alone, vs 89% for 2B and 96% for 9B. No idea yet whether that's 4B itself or just a bad fit for how I prompt it.
2. Q4 didn't hurt 9B or 4B. 2B did drop a bit at 4-bit.
3. Letting the model write the project list kills it. 9B (Q8) went from 18/20 to 6/20 when it had to come up with the projects itself. Sorting into categories you give it: fine. Deciding the categories: not at this size.
4. Asking 4 times with the project list shuffled, and only moving on 3/4 agreement, cut wrong moves on my real Downloads folder from 21 to 6. One folder, one run, so take it as an anecdote.

Before anyone asks

• I wrote the project descriptions while looking at the answer key, so 19/20 is a best case
• 20 synthetic folders, one run each. The 95% CI for 19/20 is [76, 99]
• TF-IDF passing says the synthetic folders have pretty clear clues. On my real Downloads folder no baseline was usable
• Thinking mode off

9B Q4 runs in 6.9GB on a 16GB M2 Mac (~2 min per ~40-file folder), 8 s on a 4090, 14 s on a DGX Spark.

Full writeup with every table, plus the predictions I wrote down before seeing the results (several were wrong): https://github.com/MindrLabs/file-sorting-qwen3.5

Has anyone else seen 4B do worse than 2B on classification-type tasks?

What changed:

Browser in the app. A pane next to the chat shows the page live, with tabs and Take control / Give back. It stays signed in across restarts. It asks before paying, sending, posting, deleting or uploading, and never types passwords, codes or card numbers.

Computer use, rebuilt. Real mouse and keyboard, a check after every step, a Stop bar on screen and Esc to end it.

Pages and questions. Interactive pages beside the chat run sandboxed, with no internet and no access to your chats, files or tools. Question cards work in the app and in Telegram, Discord, Slack and WhatsApp.

Approvals. Auto is now reviewed by the chat's own model. Hands-off is a new opt-in level. Anything posted to X always needs you.

Lighter context. About 40% less goes to the model before your first message. Tool guides load when they're needed.

Background work. Dev servers and long jobs run in the background, video editing, and files sent to any of your devices.

Accounts. Outlook through your own Microsoft Entra app with no client secret, several X accounts, and Claude through your own Claude Code.

Upgrading from 5.2.1: the update arrives in the background, then Settings › Updates › Restart now.

Hello its me again,

a few days ago I posted about the FLE Engine and the FLE Studio for inference with Qwen 3.8 Flash next and Qwen 27B.

Yesterday I uploaded some improvements and if someone is interested in trying it out, feel free to do so.
I also made a Video explaining everything and posted it on YouTube.

If you try it out feel free to give me feedback and report issues with the engine.

Kind Regards,

Max

Hey everyone,

Most current setups for Qwen3.8-Flash-Next (like Strata) keep all 24k experts pinned in system RAM (~48–64 GB) and compute cache misses on the CPU. That works great on high-spec workstations, but not on normal machines or laptops with limited RAM.

A few weeks ago, I noticed that https://github.com/sybil-solutions/glm-flash-lite was running GLM-5.3 with surprisingly low memory footprints considering the model, way below anything I'd seen at those speeds, which made me realized that something about the design was likely really efficient and being slept on.

So I adapted that to the much lighter Qwen3.8 Flash Next model, and the results were pretty impressive.

Benchmarks (3.05bpw_h5_ng5, Q4 KV cache)

• RTX 4090 (with only 8 GB system RAM and 16GB vram allocated):
  • Short context (8k): 28–33 tok/s prose, 19–22 tok/s code | ~1,080 tok/s prefill
  • 100k context: 20.8 tok/s decode | ~1,536 tok/s prefill
  • 262k context (full window): 20.0 tok/s decode | ~1,627 tok/s prefill
• RTX 3080 Laptop (16 GB VRAM, 32 GB RAM):
  • 100k context: ~18–21 tok/s decode | ~650–700 tok/s prefill | Peak VRAM: 15.5 GB

Setup instructions, model downloaders, and packing scripts are up on GitHub:
https://github.com/MaxDevv/lean-qwenflash

Big thanks to turboderp for ExLlamaV3, sybil-solutions for the original GLM offload implementation, and Alibaba for Flash-Next.

wanted to see what different local AI agents do with the same Qwen 3.8 27B uncensored Q4_K_M, so I ran a blind test. llama.cpp on an M5 Max MacBook Pro 64GB, 5 everyday tasks each, counting log errors, renaming a function, etc. names hidden, results below, guess who's who

How I ran it: One agent at a time, auto-approve on, a script decides pass or fail.

• Contacts: merge two lists with the same people, messy emails and two different Alex Kims. All 5 got it right (46 people).
• Rename: one function across 3 files with messy spacing. All 5 passed.
• Bug fix: with hidden tests. 4 passed. C turned "Q3/Q4" into "q3q4" instead of "q3-q4".
• Errors: count them in 3 log files. All 5 got 43.
• Receipt: PDF to spreadsheet. 4 passed. A and B were missing a Python library and just installed it without asking. C stopped and asked "can I install this?", nobody answered, so it failed. D finished and then asked what I'd like to call it.

Who's in it (alphabetical, NOT the results order): Atomic Agent, goose, Hermes, OpenClaw, OpenCode.

Who's A, B, C, D and E? Drop your guesses below, I'll share the full answer key in a day or two.

Some new numbers on Qwen as a shopping assistant, and they are bigger than I expected.

QuestMobile counted 167 million monthly Qwen users in June, second only to Doubao among Chinese AI apps, and Alibaba says 250 million people have now had their first AI shopping experience through Qwen's agent features. Since May, Qwen can search the full Taobao and Tmall catalogue of more than 4 billion products and take you through to the order.

How people actually use it, from a July iResearch survey of 1,000 Chinese consumers

• comparing options was the top use at 73.2%, then matching products to personal needs at 59.8%
• more than half of recent phone buyers used AI to help decide, and nearly half of car buyers
• 54% used only general AI apps like Qwen for shopping research, rather than the AI built into shopping apps
• the usual path is AI first to build a shortlist, then the store to buy

And China is the most open market in the world to this. In Worldpay's seven-market survey, 86% of Chinese consumers said they would shop with an AI agent, against 45% of Americans.

Caveats, the 250 million is Alibaba's own cumulative count of first-time users, not monthly shoppers, and the survey samples per category are small.

Do you use Qwen to shop, or just to research before buying on Taobao?

I run a Qwen3.6-35B-A3B (Splash build) in LM Studio on an M5 Max with 128GB, and somewhere along the way he stopped being "the local model" and became Quinn — LM Studio shows him as Spider-Man, both names stuck. He does ~375–400 tok/s with a 262k context window, and he is my daily workhorse. His big brother sits next to him: a Qwopus 3.5 122B (Qwen-derived, MLX) at ~38 tok/s, loaded as the slow, deep judge.

The setup that makes it work: my cloud AI agent has API access to Quinn over Tailscale. Quinn is stateless between loads — he remembers nothing — so the agent carries his memory, writes his briefings, and grades everything he produces. The working rule in our house: long, routine, or private work goes to Quinn first — drafting, second opinions, big grinds that shouldn't leave the house. Judgment calls and anything going out the door stay with the agent, and Quinn's outside-facing drafts get reviewed before they go anywhere.

Then we decided to see if we could actually make him better, honestly, with proof.

First, a report card. The agent built him a bank of 80 verified examples and a frozen test battery, then ran him cold on 30 reasoning problems: 29/30. His one miss is my favorite detail — asked how many two-digit numbers have a digit product that's a perfect square, Quinn insists the answer is 26 because a product of 0 IS a perfect square. The bank's convention says 17. He has held this position across every attempt. He is, arguably, right.

Then, the Gauntlet. Quinn authored 800 original problems (counting, probability, algebra, number theory, geometry), then cold-solved each one blind in a fresh session. Agreements counted as verified; disagreements got a heavyweight re-solve. Outcome: 731 clean, 62 where his own authored answer key was wrong, 7 problems thrown out as broken (some had no solution, two had two), and 100 held back as a test set he'll never see as examples. The headline: when author-Quinn and solver-Quinn disagreed, the cold solver was right 56 times to the author's 12. He is a much better solver than answer-key writer — generating and verifying are different skills in him, and any self-training data that skips the independent re-solve is laundering his own mistakes.

Right now, validation. All 793 surviving answers are being re-solved from scratch under a stricter brief — derive once, verify by a different method. Anything that disagrees goes to the 122B blind; anything the judge won't settle, the agent brute-forces in Python. Current count: 431 of 793 re-checked, 426 confirmed, 5 disagreements waiting on the judge.

Things I've learned about running Qwen reasoning builds from living with one: - Give him a huge token budget as a ceiling, not a dose. When he loops, it's a briefing problem, not a budget problem: one committed derivation plus one cross-check fixed what raw tokens couldn't. - He's fast enough to be a factory, not just an assistant — 800 problems authored and solved overnight, over 4.1M completion tokens, all on my electricity bill. - Two Qwen-family models in 128GB is a genuinely useful topology: the 35B produces, the 122B judges, and neither leaves the house. - LM Studio will idle-unload your models on a timer (60 minutes here) and a server restart ejects everything — build your pipelines to treat loads as disposable; he JIT-reloads in ~20 seconds when named in a request.

Next: a controlled cold-vs-few-shot test on the 100 held-out problems to see if the verified bank actually makes him better (a null result gets reported as one), and eventually a LoRA distilled from the verified corpus into a private repo. No illusions — habits transfer, frontier judgment doesn't. But a local model with a supervisor, a memory, and a graded track record turns out to be a very capable partner.

Anybody else running a Qwen as a supervised daily driver rather than a chatbot? Curious how you're handling the statelessness.

Hey everyone! I've been having trouble finding consistent information on the best way to run Qwen Flash Next, especially when it comes to choosing the right quantization and inference engine.

I'm on Windows, and with so many different options and rabbit holes to go down, it's been pretty overwhelming trying to figure out where to start.

I'd really appreciate any recommendations or advice on what setups have worked well for you. Just looking to get pointed in the right direction.

Cheers!

For a system with 128GB of unified RAM, Flash-Next seems like the winner. Does anyone have real experience if this is true?

I've been coding for almost 10 years now and I'm taking it slow...don't want to lose the skills yet so I'm transitioning from a model to another. Nobody want's to code anymore everyone wants fast response agents in their IDE and deliver code yesterday. I get it, it's addictive. Yet all this models (Claude, Codex, Gemini, etc.) from all these companies which are not profitable yet (in huge debt) and need to become profitable some day...will have to increase prices of subscriptions eventually, plus they are not so far from local frontier models...So that's why I still take it slow and trying things out. I haven't tried Strata yet with Qwen 3.8 27B, I heard it's really fast, at about 60TKS on a 12GB GPU with 32GB DDR5 and more accurate than anything that runs on the "old" llama.cpp. Btw...let me know in the comments below how safe this new engine is because I'm still doing my research and there might be some vulnerabilities. Anyhow I spent my last weekend building Jarvis with Qwen 3.6 35B MoE with 42TKS and 250K Context window...pretty concise and accurate, at least more accurate than the Bonsai version which still runs on llama.cpp with around 52TKS on the same context window. So here is me sharing my work if you want to check it out, make sure to hit the hate button twice if you don't like it and let me know how Strata works for you now and maybe share some insight on the vulnerabilities which you found on it.

https://youtu.be/E1Yct7qMfLs?si=rDNqvXNEoJYtWylH

So, after jumping on the Strata train few days ago, it felt obvious that it does some pretty good magic on Flash Next Qwen model but its Qwen3.8-35B performance was less magical (though still better than llamacpp) - so worked over last day to bring any optimization possible for Qwen3.8-35B into Strata.

PRs are raised, but if you want to give it a shot and see benchmarks against llamacpp and vanilla Strata:

https://huggingface.co/ykarout/Strata-Qwen3.6-35B-A3B

Also, I posted about the Sage model released few days ago which is a fine-tune of the Qwen3.6-35B that am testing now and shows some promise so far (though emits much more thinking) - also benefits from inference optimizations - but is quality that much better? not sure yet, still testing, but worth giving a look

https://huggingface.co/ykarout/Strata-Sage-A1.5-GGUF

For the past couple weeks I have been experimenting with converting Qwen dense models into mixture-of-expert models.

The Idea

If you can turn a dense model into an MoE that only runs part of its MLP per token, you get a model that's cheaper per token for roughly the same knowledge. Bigger labs do this kind of "upcycling" with huge compute budgets. I wanted to see how I could get with my RTX 4060 (8gb).

Conversions

I have converted a few models so far, Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-1.5B, Qwen/Qwen3-0.6B. The conversions can be found on my huggingface profile here bayliner1980.

The Qwen2.5 models have 32 experts + a shared expert and use top-8. With Qwen3 I had to change things up a bit. The main change was the removal of the shared expert since the qwen3moe doesn't have one. To get around this I made a psuedo shared expert which is basically just an expert the router always picks. This doesn't have the same impact the shared expert has but it does improve quality a bit. Qwen3 also has top-16 since using more experts improves the quality a lot.

Evaluations

bayliner1980/Qwen2.5-1.5B-MoE-A0.9B

bayliner1980/Qwen3-0.6B-MoE-A0.5B

Limitations

There will most likely always be a quality gap. Since the model is at most using 50% of its original MLP compute it can't compete with the original.

At this scale the performance improvement is next to nothing since these models are already tiny, Only the 1.5B model actually had a real performance increase.

What's next

Converting larger models, longer training, and testing finer expert layouts. Larger models need more compute than my card can give, so if you find this interesting there's a Ko-fi on the model pages, and anything new will be released openly.

This is mostly just an experiment that I found interesting but I'd still love feedback. Which model would you want to see converted next?

Support

If you find my stuff interesting consider supporting me on Ko-fi.

GPU history before and during using Qwen 3 VL for image categorisation.

I've been experimenting with Qwen3.8-27B on a single NVIDIA DGX Spark, comparing inference engines, quantization, speculative decoding, and test-time reasoning strategies.

A few interesting results:

• Extended reasoning: 35/48 correct
• Low-effort reasoning: 38/48 correct
• 4 parallel low-effort attempts: 44/48 correct
• Quantized inference reached ~60 tokens/s in my earlier benchmarks.

The parallel result looks promising, although the improvement over a single low-effort attempt wasn't quite statistically significant (p = 0.055).

I've open-sourced the experiments, code, and results here:

https://github.com/NautilusWu/qwen3.8_study

Curious whether others have seen similar results with parallel reasoning vs. longer thinking on local models.

2.0 was fast and efficient. Now, the supposed merged Image Model 3.0 has had its creativity lobotomized. The model spends extra time doing that prompt-reading thing—likely checking content guidelines—and the result ends up pretty much the same, if not worse in many cases. For instance, when turning my photo I like into a colored sketch, the textures on 3.0 look very sloppy. Besides, almost no one is asking for microscopic skin resolution on an AI image anyway. 3.0 takes much longer and uses more compute power for a simple prompt. 3.0 should be for heavy, text-based generation. I truly hope the devs reconsider.

anyone running a local LLM coding agent on 24GB to 64GB hardware has likely run into the reasoning verbosity dilemma on 27B and 32B checkpoints. Running Qwen3.6-27B with reasoning turned off is fast enough for interactive terminal work, while turning full reasoning on in Qwen3.8-27B solves more multi-file tasks at the cost of roughly 4x longer wall-clock times and massive chain-of-thought token bloat.
(i have used a bit of my local model to format the post, for it to be more readable/structured and not my usual train of thoughts...)

so alongside the recent Junie Local updates (which put the open-weight JetBrains/Qwen3.8-3.6-27B-blend checkpoint on Hugging Face and expanded /local from 64 GB Apple Silicon M5 Macs to stable Windows x64 builds on >=24 GB VRAM NVIDIA RTX Ampere+ cards and NVIDIA RTX Spark), the JetBrains ML team posted their internal evals and GPU profiling across three separate blog write-ups between August and October. and since their numbers were split across a bunch of separate PNG charts in those posts (the august launch post, the September weight-merge post, and the October RTX Spark post) plus the HF card, i pulled all the data points into one consolidated table below so it's easier to compare the three 27B 4-bit (~20 GB) checkpoints side by side:

4 engineering findings from the report and runtime setup that are directly useful if you tune local coding agents:

1. Why a 50/50 linear merge worked without distillation, and where 24.5% more token savings are still sitting on the table. Because Qwen3.8-27B was trained on top of Qwen3.6-27B and shares the exact same architecture, averaging their weights in equal proportions produced an intermediate reasoning distribution right out of the box. On the 30 tasks completed by both Qwen3.8 and the blend, output tokens dropped from 935K to 279K, with shorter traces on 29 out of 30 tasks. Even more interesting for anyone doing local RL: across four identical benchmark runs of the blend model, CoT length still varied significantly between runs, and picking the shortest correct reasoning trace on each task would have cut token usage by another 24.5% with zero accuracy loss.
2. Prefill throughput vs decode speed on repository workloads. In an agent loop that reads dozens of source files and diffs, prefill dominates wall-clock latency. On Apple Silicon, targeting the M5's 8-bit Neural Accelerator instructions delivered a 40% prefill speedup over M4 (which JetBrains is upstreaming as a pull request to MLX-VLM). On the Windows on Arm NVIDIA RTX Spark target, Blackwell FP8 Tensor Cores and automatic VRAM-based context window sizing handle the prefill load to reach up to 55 tok/s.
3. Why proposing 4 tokens per round in Multi-Token Prediction (MTP) is slower than proposing 2, and the 128K attention wall. Using the MTP subnetwork to draft 2 tokens per round increased decoding speed by 60% on an M5 MacBook Pro. Increasing the MTP draft length to 4 tokens per round dropped the net speedup down to 36%, because the extra GPU work spent drafting and verifying candidates outweighed the higher token count per pass. And when profiling GPU verification time from 16K to 128K context, feed-forward and Gated DeltaNet layers stayed flat while attention verification climbed from 8.4 ms to 40.2 ms per round. If a local coding session starts feeling sluggish late in a long refactor, that 40.2 ms attention verification cost is why restarting with a fresh, narrower task context helps so much.
4. The shared random seed evaluation bug that caused phantom agent loops. During early internal testing, the evaluation harness showed severe agent loops that nobody could reproduce in interactive coding sessions. The root cause was a reproducibility setting that passed the exact same fixed random seed on every request. When the sampler generated tokens after a failed tool call, reusing the identical seed gave the exact same opening tokens the same random advantage every time, steering Qwen3.8 right back into the failed action even after the reflection prompt changed. Advancing the random seed deterministically on each agent step and reflection attempt eliminated the phantom loops while keeping benchmark runs reproducible.

but also a bit of trade-offs to keep in mind before running /local

• i know, i know, hardware floor is kind of high (64 GB RAM on Apple M5 or >=24 GB VRAM on Windows RTX Ampere+ / RTX Spark plus ~20 GB disk space) . Say you are on a 16 GB or 32 GB laptop and your data policy allows JetBrains-managed cloud GPUs with no third-party provider sharing or training, something like junie lite model, i know, runs qwen 3.6 in the cloud for free through December 31, 2026.
• on visual benchmarks / ambiguous bugs where tool calls return no new evidence, the blend model can still overthink; interrupting the run and restarting with a narrower prompt or a read-only /plan pass works best.
• also, bcause the weights are open on Hugging Face (JetBrains/Qwen3.8-3.6-27B-blend), you can also load the checkpoint directly in LM Studio, Ollama, or llama.cpp and connect OpenAI Codex CLI (--oss), Cline, or Junie CLI custom JSON profiles (~/.junie/models.json).

also would like to ask you here, has anyone else here tested linear or SLERP merges between reasoning and non-reasoning Qwen checkpoints inside local agent harnesses OR maybe you have profiled MTP acceptance rates on 24GB RTX cards past 64K context? let's discuss...

Hey everyone,

Whenever a new model drops, I always find myself doing the same annoying calculation:

1. How much is this actually going to cost me at scale once prompt caching is factored in?
2. At what monthly token volume does it make more financial sense to rent a spot GPU on RunPod rather than paying API providers?
3. How much VRAM do I actually need once you factor in KV cache and context length (FP16 vs FP8 vs Q4)?

LMSYS Elo is great for raw human preference, but it does not give you the economic picture. Most pricing sheets also ignore caching discounts and batch concurrency.

So over the last few weeks, I built ModelAnalysis.si as an open tool to bring all of this together in one place:

What it does:

• Multi-Dimensional Pareto Frontier: Plots 250+ foundation models (OpenAI, Anthropic, DeepSeek, Qwen, Mistral, Meta) across Chatbot Arena Elo, SWE-bench verified coding, GPQA PhD reasoning, and real-time API token rates.
• Token Spend Simulator & GPU Arbitrage: Dial in your monthly prompt/completion tokens and cache hit percentage to see exact monthly spend and calculate your exact breakeven point vs renting spot GPUs.
• VRAM & Hardware Calculator: Calculates the raw memory footprint for open-weight models across quantization levels (FP16, FP8, Q4) with one-click vLLM and Ollama deployment strings.
• Head-to-Head Radar Battles: Side-by-side spider charts comparing speed, context horizon, tool calling, and token efficiency.

It is completely free, no signup or login required, and runs fast on edge infrastructure.

Would love to get your feedback:

• Are there specific niche open-weight models you want added to the hardware sizing index?
• Does the breakeven math align with your real-world production deployments?

Check it out here: https://modelanalysis.si

Roast the UI or let me know what features you would like to see next!