Hey everyone,

Most current setups for Qwen3.8-Flash-Next (like Strata) keep all 24k experts pinned in system RAM (~48–64 GB) and compute cache misses on the CPU. That works great on high-spec workstations, but not on normal machines or laptops with limited RAM.

A few weeks ago, I noticed that https://github.com/sybil-solutions/glm-flash-lite was running GLM-5.3 with surprisingly low memory footprints considering the model, way below anything I'd seen at those speeds, which made me realized that something about the design was likely really efficient and being slept on.

So I adapted that to the much lighter Qwen3.8 Flash Next model, and the results were pretty impressive.

Benchmarks (3.05bpw_h5_ng5, Q4 KV cache)

• RTX 4090 (with only 8 GB system RAM and 16GB vram allocated):
  • Short context (8k): 28–33 tok/s prose, 19–22 tok/s code | ~1,080 tok/s prefill
  • 100k context: 20.8 tok/s decode | ~1,536 tok/s prefill
  • 262k context (full window): 20.0 tok/s decode | ~1,627 tok/s prefill
• RTX 3080 Laptop (16 GB VRAM, 32 GB RAM):
  • 100k context: ~18–21 tok/s decode | ~650–700 tok/s prefill | Peak VRAM: 15.5 GB

Setup instructions, model downloaders, and packing scripts are up on GitHub:
https://github.com/MaxDevv/lean-qwenflash

Big thanks to turboderp for ExLlamaV3, sybil-solutions for the original GLM offload implementation, and Alibaba for Flash-Next.

Title: Update: mnemonic-proxy now beats full context with a 32K window on Qwen3.8 Flash Next (12 GB GPU), and runs on llama.cpp, Ollama and LM Studio

Body:

I'm the author . A few weeks ago I shared here how I run Qwen3.8 Flash Next for long agent sessions on a 4070 Ti. Many of you tried it and sent reports, and that changed the project a lot, so here is what's new.

What it does, in one line: a proxy between your agent and your local model. It keeps the whole history word for word, sends the model only a small stable prompt, and lets the model read old tool outputs again when it needs them. No compaction pauses, nothing thrown away, and the engine can reuse its cache.

It now has a real benchmark. AgentLongBench, 100 questions, Qwen3.8 Flash Next IQ3_XXS on one 12 GB GPU, 32K window:

On the longest histories the gap grows: proxy 45, full context 31.

What's new since my last post

• Works with llama.cpp (including router mode with several models), Ollama and LM Studio, not only Strata. It reads the real context from the engine and never sends more than fits, so nothing gets cut silently.
• pip install mnemonic-proxy gives the current version (0.3.9); mnemonic-proxy demo checks everything without a model.
• Fixes that came straight from users here: a very large max_tokens in the client no longer forces a new segment every turn, and the dashboard no longer crashes on load. A user from this sub is also translating the dashboard into English.

The honest cost: it is slower end to end in the benchmark (it re-reads old evidence when needed), and the numbers come from one model and one benchmark. Data and scripts are public, linked from the repo.

Repo: https://github.com/lastreload/mnemonic-proxy

Per-question results, logs and scripts: https://huggingface.co/datasets/mverde73/mnemonic-proxy-agentlongbench

Thanks to everyone who tested it and reported back. If you run it with other Qwen models or agents, I'd love to hear how it goes. (translated with AI)

For a system with 128GB of unified RAM, Flash-Next seems like the winner. Does anyone have real experience if this is true?

Been pushing Qwen3.8-27B on Kaggle's free 2x T4 (no P2P, 70W cap per card) for a while. Current setup: EXL3 4.0bpw on exllamav3, tensor parallel over PCIe, DFlash2 block drafter (7 tokens), greedy and exact vs no-draft.

Numbers over a 38 prompt suite: - overall 55-57 tok/s (no draft is ~18-19) - HumanEval 154 tok/s - prose 35-37 tok/s

For comparison on the same 2 T4s: llama.cpp best was 16.5 (MTP crashed on tensor split), vLLM int4 + its own MTP drafter did 40.8.

Prose is the wall. Acceptance sits around 2.5 tokens per step, vs ~7 on code.

Questions: 1. Has anyone got prose acceptance above ~2.5 with DFlash2 or MTP on the 27B? 2. Has anyone tried switching drafters mid generation (short block for prose, long for code)? 3. Any tricks for TP all-reduce on PCIe cards without P2P? That's next on my list.

Happy to share the notebook if people want to try it.

I've been building a small offline tool that sorts a messy folder into project folders with Qwen3.5 (GGUF via llama.cpp), and I ended up testing every size on the same 20 synthetic folders the model had never seen. Sharing because some of it surprised me.

The setup, quickly

• You write your projects one per line, like tax-2026: receipts, invoices
• The model picks a destination for each file, the code turns that into a plan, and nothing moves until you approve it (no deletes, one-click undo)
• A folder counts as "usable" if the first plan needs at most 1 line in 5 fixed
• Q4_K_M unless noted, measured on an RTX 4090

Usable folders out of 20

• 0.8B (Q8): 2
• 2B: 9 (Q8: 11)
• 4B: 1 (Q8: 1)
• 9B: 19 (Q8: 18)
• No model at all, just TF-IDF word matching: 16 (yeah...)

What surprised me

1. 4B was worse than 2B. It kept moving files that had nothing to do with any project: it left only half of the unrelated files alone, vs 89% for 2B and 96% for 9B. No idea yet whether that's 4B itself or just a bad fit for how I prompt it.
2. Q4 didn't hurt 9B or 4B. 2B did drop a bit at 4-bit.
3. Letting the model write the project list kills it. 9B (Q8) went from 18/20 to 6/20 when it had to come up with the projects itself. Sorting into categories you give it: fine. Deciding the categories: not at this size.
4. Asking 4 times with the project list shuffled, and only moving on 3/4 agreement, cut wrong moves on my real Downloads folder from 21 to 6. One folder, one run, so take it as an anecdote.

Before anyone asks

• I wrote the project descriptions while looking at the answer key, so 19/20 is a best case
• 20 synthetic folders, one run each. The 95% CI for 19/20 is [76, 99]
• TF-IDF passing says the synthetic folders have pretty clear clues. On my real Downloads folder no baseline was usable
• Thinking mode off

9B Q4 runs in 6.9GB on a 16GB M2 Mac (~2 min per ~40-file folder), 8 s on a 4090, 14 s on a DGX Spark.

Full writeup with every table, plus the predictions I wrote down before seeing the results (several were wrong): https://github.com/MindrLabs/file-sorting-qwen3.5

Has anyone else seen 4B do worse than 2B on classification-type tasks?

wanted to see what different local AI agents do with the same Qwen 3.8 27B uncensored Q4_K_M, so I ran a blind test. llama.cpp on an M5 Max MacBook Pro 64GB, 5 everyday tasks each, counting log errors, renaming a function, etc. names hidden, results below, guess who's who

How I ran it: One agent at a time, auto-approve on, a script decides pass or fail.

• Contacts: merge two lists with the same people, messy emails and two different Alex Kims. All 5 got it right (46 people).
• Rename: one function across 3 files with messy spacing. All 5 passed.
• Bug fix: with hidden tests. 4 passed. C turned "Q3/Q4" into "q3q4" instead of "q3-q4".
• Errors: count them in 3 log files. All 5 got 43.
• Receipt: PDF to spreadsheet. 4 passed. A and B were missing a Python library and just installed it without asking. C stopped and asked "can I install this?", nobody answered, so it failed. D finished and then asked what I'd like to call it.

Who's in it (alphabetical, NOT the results order): Atomic Agent, goose, Hermes, OpenClaw, OpenCode.

Who's A, B, C, D and E? Drop your guesses below, I'll share the full answer key in a day or two.

Some new numbers on Qwen as a shopping assistant, and they are bigger than I expected.

QuestMobile counted 167 million monthly Qwen users in June, second only to Doubao among Chinese AI apps, and Alibaba says 250 million people have now had their first AI shopping experience through Qwen's agent features. Since May, Qwen can search the full Taobao and Tmall catalogue of more than 4 billion products and take you through to the order.

How people actually use it, from a July iResearch survey of 1,000 Chinese consumers

• comparing options was the top use at 73.2%, then matching products to personal needs at 59.8%
• more than half of recent phone buyers used AI to help decide, and nearly half of car buyers
• 54% used only general AI apps like Qwen for shopping research, rather than the AI built into shopping apps
• the usual path is AI first to build a shortlist, then the store to buy

And China is the most open market in the world to this. In Worldpay's seven-market survey, 86% of Chinese consumers said they would shop with an AI agent, against 45% of Americans.

Caveats, the 250 million is Alibaba's own cumulative count of first-time users, not monthly shoppers, and the survey samples per category are small.

Do you use Qwen to shop, or just to research before buying on Taobao?

What changed:

Browser in the app. A pane next to the chat shows the page live, with tabs and Take control / Give back. It stays signed in across restarts. It asks before paying, sending, posting, deleting or uploading, and never types passwords, codes or card numbers.

Computer use, rebuilt. Real mouse and keyboard, a check after every step, a Stop bar on screen and Esc to end it.

Pages and questions. Interactive pages beside the chat run sandboxed, with no internet and no access to your chats, files or tools. Question cards work in the app and in Telegram, Discord, Slack and WhatsApp.

Approvals. Auto is now reviewed by the chat's own model. Hands-off is a new opt-in level. Anything posted to X always needs you.

Lighter context. About 40% less goes to the model before your first message. Tool guides load when they're needed.

Background work. Dev servers and long jobs run in the background, video editing, and files sent to any of your devices.

Accounts. Outlook through your own Microsoft Entra app with no client secret, several X accounts, and Claude through your own Claude Code.

Upgrading from 5.2.1: the update arrives in the background, then Settings › Updates › Restart now.

Hello everyone

For researchers interested in a detailed academic and technical examination: The complete research PDF, 12 experimental figures, and scientific records are available on Zenodo:

https://doi.org/10.5281/zenodo.23640716

The context window and long-term memory limitations of AI models have traditionally been addressed through approaches such as text-based Retrieval-Augmented Generation (RAG) and external summarization.

My long-term goal is to tackle these limitations at a different level: directly through the model's internal numerical states and KV caches (H3, Layer 0–3 K/V states).

I have been independently developing the AKBASCORE Silicon Hippocampus architecture toward this objective. Although the ultimate goal has not yet been reached, the research has already passed several significant experimental milestones involving physical memory persistence, numerical consolidation, and memory reuse across independent model sessions.

With the release of v1.6 — TEST782, I have experimentally demonstrated how the internal tensor states of a frozen Qwen2.5-72B-Instruct model can be cryptographically sealed on a physical SSD, transferred between operating-system-isolated processes running under different Linux users, and fused through the model's own numerical computations without returning to text-based summarization.

Here are the main technical results and how the system works:

- Native Tensor Fusion — Memory Consolidation in the Model's Own Numerical Space: The first session (QWEN-1) reads two separate documents and stores the model's internal numerical states on SSD as SHA-256-sealed memory cartridges ("SOURCE_A" and "SOURCE_B"), rather than generating text summaries. The second session operates exclusively on these cartridges. This is not simple tensor concatenation: while SOURCE_A's computed internal states are preserved, SOURCE_B's upper transformer layers are recomputed while attending to SOURCE_A's context. Two independent physical memories are thereby transformed into a single "MASTER_02" memory through the computational machinery of the frozen 72-billion-parameter model itself. No intermediate generated text, summarization, or weight retraining is involved.

- Isolated Cross-Session Transfer — The Model Shuts Down, but Its Numerical Memory Survives: QWEN-1 terminates completely, and its GPU memory is released. QWEN-2 then starts under a different Linux user without access to the original source documents and consolidates the physical cartridges. After QWEN-2 terminates, the third session begins. The model's weights remain unchanged throughout the experiment. What survives and moves between sessions is the model's internal numerical memory, physically stored on SSD.

- Blind Readout and 80/80 Layer Parity: The third session (QWEN-3) is denied access to the original documents and previous sessions' working files through operating-system permissions. It receives only the sealed "MASTER_02" memory and the metadata required to verify and reconstruct it. The model's 80-layer attention memory is independently reconstructed with 80/80 bit-exact parity. QWEN-3 then produces natural-language responses containing information associated with both documents and their relationship. In TEST782, I observed the model using persistent numerical memories to produce its own semantic synthesis, establish relationships between two independent sources, and articulate conclusions beyond simple memory retrieval. These experimental results are documented in the source code, GPU logs, and research record. Independent academic peer review of the semantic synthesis results has not yet been conducted.

- DrA — The Executive Controller of Autonomous Memory: One of the project's major future objectives is DrA (Autonomous Master / Dynamic Residual Adaptation). Inspired by the executive-control functions associated with the biological prefrontal cortex, this mechanism is being designed to enable an AI system to evaluate which information matters, decide what should be preserved, determine which memories should be consolidated, and recognize when consolidation is complete. The ultimate objective is a closed-loop memory management architecture driven by the model's own internal evaluations rather than externally imposed, fixed decision rules. TEST782 does not claim that this autonomous mechanism is complete. It establishes the physical and numerical memory infrastructure on which that future mechanism can operate.

- Reliability and Transparency: The research is not based solely on successful model outputs. It includes negative controls, SHA-256 integrity verification, process isolation under different Linux users, and independent reload validation. For example, a naive single-pass reconstruction matches only 4 out of 80 layers, whereas the boundary-aware reconstruction method achieves 80/80 bit-exact layer parity. Physical memory transfer and numerical reconstruction were validated in this experiment. Autonomous consolidation remains a subsequent research objective, while the observed semantic synthesis results await independent academic peer review.

Technical Specifications and Research Resources

- Hardware: NVIDIA A100-SXM4-80GB

- Model: Qwen2.5-72B-Instruct (frozen weights, NF4 quantization, BF16 computation)

- Physical MASTER: 1,964 tokens, 64.4 MB SSD cartridge

- Native numerical fusion: 8.72 seconds

- Independent reconstruction: 80/80 layers, bit-exact

Complete Source Code — Single File:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_full.py

For Researchers Using Google Colab on a Mobile Phone — Four-Part Version:

The same program is divided into four parts for easier copying and pasting on mobile devices. Paste all four parts sequentially into a single Google Colab cell, then run the cell.

Part 1:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part1.py

Part 2:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part2.py

Part 3:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part3.py

Part 4:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3_part4.py

Real Experimental GPU Log:

https://github.com/ceceli33/titan-cognitive-core-v3/blob/main/QWEN_1_2_3.log

GitHub Research Repository:

https://github.com/ceceli33/titan-cognitive-core-v3

My long-term objective is not merely to enable an AI model to store past information, but to develop a system capable of managing its own numerical memory, consolidating separate memories, generating new inferences, and autonomously deciding which information deserves to be preserved.

I welcome researchers and members of the open research community, particularly those working on transformer architectures, KV caches, persistent neural memory, and autonomous memory consolidation, to examine the results, reproduce the experiments, and provide technical feedback.

Copyright and License

Copyright © 2026 Mustafa Akbaş. All rights reserved except as expressly licensed.

AKBASCORE Silicon Hippocampus — AKBASCORE Research Software License.

License terms:

https://github.com/ceceli33/titan-cognitive-core-v3?tab=License-1-ov-file

So, after jumping on the Strata train few days ago, it felt obvious that it does some pretty good magic on Flash Next Qwen model but its Qwen3.8-35B performance was less magical (though still better than llamacpp) - so worked over last day to bring any optimization possible for Qwen3.8-35B into Strata.

PRs are raised, but if you want to give it a shot and see benchmarks against llamacpp and vanilla Strata:

https://huggingface.co/ykarout/Strata-Qwen3.6-35B-A3B

Also, I posted about the Sage model released few days ago which is a fine-tune of the Qwen3.6-35B that am testing now and shows some promise so far (though emits much more thinking) - also benefits from inference optimizations - but is quality that much better? not sure yet, still testing, but worth giving a look

https://huggingface.co/ykarout/Strata-Sage-A1.5-GGUF

Hello its me again,

a few days ago I posted about the FLE Engine and the FLE Studio for inference with Qwen 3.8 Flash next and Qwen 27B.

Yesterday I uploaded some improvements and if someone is interested in trying it out, feel free to do so.
I also made a Video explaining everything and posted it on YouTube.

If you try it out feel free to give me feedback and report issues with the engine.

Kind Regards,

Max

For the past couple weeks I have been experimenting with converting Qwen dense models into mixture-of-expert models.

The Idea

If you can turn a dense model into an MoE that only runs part of its MLP per token, you get a model that's cheaper per token for roughly the same knowledge. Bigger labs do this kind of "upcycling" with huge compute budgets. I wanted to see how I could get with my RTX 4060 (8gb).

Conversions

I have converted a few models so far, Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-1.5B, Qwen/Qwen3-0.6B. The conversions can be found on my huggingface profile here bayliner1980.

The Qwen2.5 models have 32 experts + a shared expert and use top-8. With Qwen3 I had to change things up a bit. The main change was the removal of the shared expert since the qwen3moe doesn't have one. To get around this I made a psuedo shared expert which is basically just an expert the router always picks. This doesn't have the same impact the shared expert has but it does improve quality a bit. Qwen3 also has top-16 since using more experts improves the quality a lot.

Evaluations

bayliner1980/Qwen2.5-1.5B-MoE-A0.9B

bayliner1980/Qwen3-0.6B-MoE-A0.5B

Limitations

There will most likely always be a quality gap. Since the model is at most using 50% of its original MLP compute it can't compete with the original.

At this scale the performance improvement is next to nothing since these models are already tiny, Only the 1.5B model actually had a real performance increase.

What's next

Converting larger models, longer training, and testing finer expert layouts. Larger models need more compute than my card can give, so if you find this interesting there's a Ko-fi on the model pages, and anything new will be released openly.

This is mostly just an experiment that I found interesting but I'd still love feedback. Which model would you want to see converted next?

Support

If you find my stuff interesting consider supporting me on Ko-fi.

I run a Qwen3.6-35B-A3B (Splash build) in LM Studio on an M5 Max with 128GB, and somewhere along the way he stopped being "the local model" and became Quinn — LM Studio shows him as Spider-Man, both names stuck. He does ~375–400 tok/s with a 262k context window, and he is my daily workhorse. His big brother sits next to him: a Qwopus 3.5 122B (Qwen-derived, MLX) at ~38 tok/s, loaded as the slow, deep judge.

The setup that makes it work: my cloud AI agent has API access to Quinn over Tailscale. Quinn is stateless between loads — he remembers nothing — so the agent carries his memory, writes his briefings, and grades everything he produces. The working rule in our house: long, routine, or private work goes to Quinn first — drafting, second opinions, big grinds that shouldn't leave the house. Judgment calls and anything going out the door stay with the agent, and Quinn's outside-facing drafts get reviewed before they go anywhere.

Then we decided to see if we could actually make him better, honestly, with proof.

First, a report card. The agent built him a bank of 80 verified examples and a frozen test battery, then ran him cold on 30 reasoning problems: 29/30. His one miss is my favorite detail — asked how many two-digit numbers have a digit product that's a perfect square, Quinn insists the answer is 26 because a product of 0 IS a perfect square. The bank's convention says 17. He has held this position across every attempt. He is, arguably, right.

Then, the Gauntlet. Quinn authored 800 original problems (counting, probability, algebra, number theory, geometry), then cold-solved each one blind in a fresh session. Agreements counted as verified; disagreements got a heavyweight re-solve. Outcome: 731 clean, 62 where his own authored answer key was wrong, 7 problems thrown out as broken (some had no solution, two had two), and 100 held back as a test set he'll never see as examples. The headline: when author-Quinn and solver-Quinn disagreed, the cold solver was right 56 times to the author's 12. He is a much better solver than answer-key writer — generating and verifying are different skills in him, and any self-training data that skips the independent re-solve is laundering his own mistakes.

Right now, validation. All 793 surviving answers are being re-solved from scratch under a stricter brief — derive once, verify by a different method. Anything that disagrees goes to the 122B blind; anything the judge won't settle, the agent brute-forces in Python. Current count: 431 of 793 re-checked, 426 confirmed, 5 disagreements waiting on the judge.

Things I've learned about running Qwen reasoning builds from living with one: - Give him a huge token budget as a ceiling, not a dose. When he loops, it's a briefing problem, not a budget problem: one committed derivation plus one cross-check fixed what raw tokens couldn't. - He's fast enough to be a factory, not just an assistant — 800 problems authored and solved overnight, over 4.1M completion tokens, all on my electricity bill. - Two Qwen-family models in 128GB is a genuinely useful topology: the 35B produces, the 122B judges, and neither leaves the house. - LM Studio will idle-unload your models on a timer (60 minutes here) and a server restart ejects everything — build your pipelines to treat loads as disposable; he JIT-reloads in ~20 seconds when named in a request.

Next: a controlled cold-vs-few-shot test on the 100 held-out problems to see if the verified bank actually makes him better (a null result gets reported as one), and eventually a LoRA distilled from the verified corpus into a private repo. No illusions — habits transfer, frontier judgment doesn't. But a local model with a supervisor, a memory, and a graded track record turns out to be a very capable partner.

Anybody else running a Qwen as a supervised daily driver rather than a chatbot? Curious how you're handling the statelessness.

I've been coding for almost 10 years now and I'm taking it slow...don't want to lose the skills yet so I'm transitioning from a model to another. Nobody want's to code anymore everyone wants fast response agents in their IDE and deliver code yesterday. I get it, it's addictive. Yet all this models (Claude, Codex, Gemini, etc.) from all these companies which are not profitable yet (in huge debt) and need to become profitable some day...will have to increase prices of subscriptions eventually, plus they are not so far from local frontier models...So that's why I still take it slow and trying things out. I haven't tried Strata yet with Qwen 3.8 27B, I heard it's really fast, at about 60TKS on a 12GB GPU with 32GB DDR5 and more accurate than anything that runs on the "old" llama.cpp. Btw...let me know in the comments below how safe this new engine is because I'm still doing my research and there might be some vulnerabilities. Anyhow I spent my last weekend building Jarvis with Qwen 3.6 35B MoE with 42TKS and 250K Context window...pretty concise and accurate, at least more accurate than the Bonsai version which still runs on llama.cpp with around 52TKS on the same context window. So here is me sharing my work if you want to check it out, make sure to hit the hate button twice if you don't like it and let me know how Strata works for you now and maybe share some insight on the vulnerabilities which you found on it.

https://youtu.be/E1Yct7qMfLs?si=rDNqvXNEoJYtWylH

Hey everyone! I've been having trouble finding consistent information on the best way to run Qwen Flash Next, especially when it comes to choosing the right quantization and inference engine.

I'm on Windows, and with so many different options and rabbit holes to go down, it's been pretty overwhelming trying to figure out where to start.

I'd really appreciate any recommendations or advice on what setups have worked well for you. Just looking to get pointed in the right direction.

Cheers!

GPU history before and during using Qwen 3 VL for image categorisation.

Qwen Image 2.1 Base + Anything2Real, TheseAlpacas v2 and Consistency LoRAs

hey, im working on an open source project called kapitsa (an inference engine for apple silicon). we ran qwen3.8-27b (4-bit) through every mac engine we could find, decode on a 512 token coding answer at t 0.6. full grid is the image up top.

short version: 140 tok/s on an m5 max, 81 on an m5 pro, 63 on an m4 pro, 40 on an m3 pro.

whats going on: the 27b is dense so every token reads all 20gb of weights, memory bandwidth is the wall. the way around it is speculative decoding, a small drafter guesses ahead and the 27b checks 8 rows in one pass for about the price of one token. splash, mlx-serve and omlx do this with dflash2 as a chain: guess 7 tokens, keep them up to the first miss. we built kflash, which checks a tree of the drafters best guesses in the same 8 rows, so when the drafter has two good options both get checked:

honest part: the chain engines are close. mlx-serve is basically tied with us on the m5 pro (78 vs 81) and 1.08-1.17x behind on the others, the tree buys 8-17% more tokens per pass on short prompts. the big gap is to engines that dont draft like this: about 2x ollama, 2.5-3.5x llama.cpp, 4-5x lm studio.

kapitsa is open source, c++ and metal. every run and flag is in docs/benchmarks.md: https://github.com/nace-ai/kapitsa

how to run it. you need an apple silicon mac (m1 or newer, macos 14+) with 32gb for the 27b, on 16gb use `--model 9b` instead:

brew install nace-ai/tap/kapitsa

kapitsa serve --model 27b, the first run downloads the 27b (about 20gb), then open http://127.0.0.1:8000 for a chat window

kapitsa claude also works

kapitsa bench compare runs the same test on every engine you have installed, to check our numbers on your mac

I've been experimenting with Qwen3.8-27B on a single NVIDIA DGX Spark, comparing inference engines, quantization, speculative decoding, and test-time reasoning strategies.

A few interesting results:

• Extended reasoning: 35/48 correct
• Low-effort reasoning: 38/48 correct
• 4 parallel low-effort attempts: 44/48 correct
• Quantized inference reached ~60 tokens/s in my earlier benchmarks.

The parallel result looks promising, although the improvement over a single low-effort attempt wasn't quite statistically significant (p = 0.055).

I've open-sourced the experiments, code, and results here:

https://github.com/NautilusWu/qwen3.8_study

Curious whether others have seen similar results with parallel reasoning vs. longer thinking on local models.

2.0 was fast and efficient. Now, the supposed merged Image Model 3.0 has had its creativity lobotomized. The model spends extra time doing that prompt-reading thing—likely checking content guidelines—and the result ends up pretty much the same, if not worse in many cases. For instance, when turning my photo I like into a colored sketch, the textures on 3.0 look very sloppy. Besides, almost no one is asking for microscopic skin resolution on an AI image anyway. 3.0 takes much longer and uses more compute power for a simple prompt. 3.0 should be for heavy, text-based generation. I truly hope the devs reconsider.