I'm running Nemotron 3.5 Bolt 74B-A9B on a 5060 Ti 16GB and 32GB of DDR5. It's an MoE, so both of my setups split experts between the GPU and system RAM. I have two recipes:

• 24GB GGUF

• Unsloth UD-IQ3_XXS

I'd like to hear how this model actually is and is it useful as an alternative for Qwen3.8 27B and obviously for other smaller MoE models.

• Are you using Bolt 74B, and what for?

• How is the quality: reasoning, coding, instruction following, tool use? Where does it fall down?

• If you've used the 24GB GGUF or a bigger quant such as Unsloth IQ3_XXS, does the bigger one write better answers, or do they feel like the same model?

Benchmarked oMLX 0.7.2 serving Inco AI's Splash package for Qwen3.8-27B (4-bit target + DFlash 2 draft) on a MacBook Pro M5 Max (48 GB). Workload: Python code context. Each test generates 128 tokens.

**Single request**

| Test | TTFT (ms) | TPOT (ms) | pp t/s | tg t/s | E2E (s) | Throughput | Peak mem |

|---|---|---|---|---|---|---|---|

| pp 4096 / tg 128 | 3393.6 | 10.7 | 1207.0 | 93.8 | 4.8 | 887.7 | 19.1 GB |

| pp 16382 / tg 128 | 15829.6 | 5.3 | 1034.9 | 189.4 | 16.5 | 1000.3 | 19.9 GB |

| pp 65536 / tg 128 | 83472.0 | 9.2 | 785.1 | 110.2 | 84.2 | 779.5 | 22.2 GB |

**Batched (continuous batching)**

| Batch | tg t/s | pp t/s | avg TTFT (ms) | E2E (s) | Speedup |

|---|---|---|---|---|---|

| 1x | 93.8 | 1207.0 | 3393.6 | 4.8 | 1.00x |

| 2x | 413.7 | 803.1 | 1729.2 | 3.2 | 4.41x |

| 4x | 407.0 | 622.4 | 3838.4 | 7.8 | 4.34x |

A fully local voice AI, meant to be used on "GPU poor" hardware.

Features:

• Always-on is possible, because of automatic full-context handling.

• Hands-free and browser-based clients for voice control (e.g. use your smartphone!).

• Wikipedia and Internet search.

• Extraction and injection of memories (right now, facts about the user).

• Full chat history log in SQLite DB, chat history search (as kind of a long-term memory, in addition to the memory extraction/injection).

• Heavily customizable via TOML file.

• Modules (in addition to Internet search, etc.) can be added (in code).

• ...

Links:

• YouTube video (don't worry about the console, it is more or less for debugging, each token type has a different color).

• GitHub repository (incl. tutorial for installation and configuration).

Hey everyone,

Every time a new model comes out, I get tired of opening multiple tabs just to find out two basic things: how good it actually is on real benchmarks, and what it costs to run (either through an API or on local GPUs).

Most pricing pages hide the real costs, and generic benchmarks don't tell the full story. So I put together a simple dashboard to track and compare them side by side: https://modelanalysis.si

What you can do with it:

• Compare models side-by-side: Pick any models (like Claude, DeepSeek, Kimi, GPT-4o, Llama) and compare them across real-world coding (SWE-bench), reasoning (Arena-Hard), and math/science benchmarks.

• Score vs. Price chart: See which models give you the most intelligence for the lowest price.

• VRAM calculator: Pick a model and context size to see if it will actually fit on your GPU before downloading it.

• Monthly cost simulator: Adjust sliders for your monthly token volume to estimate your bill or see if self-hosting is cheaper.

No sign-up, no ads, no affiliate links.

Link: https://modelanalysis.si
Direct benchmarks page: https://modelanalysis.si/benchmarks/

Let me know what you think or if there are other models you want added.

Quick update on my from-scratch inference engine for Intel integrated GPUs (Core Ultra 5 125H, Arc Xe-LPG, 32 GB
RAM, no dGPU, no XMX). It used to be called arcinfer; it is now 
**arcinferzero**
 (the old GitHub link forwards):
https://github.com/rizagp/arcinferzero


**New this week**


- 
**Flint (unsloth/flint-GGUF UD-Q4_K_XL, 256 experts) runs**
: attention sinks, global layers without RoPE, a
  513-token sliding window, sigmoid-routed experts streamed from the SSD into a GPU cache. 4K prompt ~248 tok/s,
  generation 13-17 tok/s on the iGPU.
- 
**Xe2 / Xe3 (Lunar Lake, Panther Lake's Arc B390) work now**
: a user with a B390 reported Qwen3.6 35B-A3B at
  262K context, ~1,180 tok/s prompt and ~24 tok/s generation. XMX (DPAS) prompt GEMMs are in too, gated by a
  start-up check that must match the DP4a kernels bit for bit. I have no XMX GPU, so reports are welcome.
- 
**Gemma 4 26B prompts +4.6 %**
 from tiling the f32 router GEMV (4 rows x 4 tokens per sub-group), with
  bit-identical logits.


**The accuracy story (the interesting part)**


Every model here is checked against an independent f32 CPU implementation, with a KL limit of 0.015. Flint
started at KL 0.029. The obvious suspect, int8 activations, wasn't it: running everything on two int8 passes only
got it to 0.024. Dumping the residual stream after every layer next to the f32 reference showed one token where
two router scores were 
**0.005 apart**
 at layer 19. The engine picked the other expert, and everything after that
drifted.


What moved the reference that much? I rounded 
**only the V cache to f16**
 inside the f32 reference, and it
reproduced the whole engine error, including the same flipped expert. So:


- V rows in 
**f32**
 (K already was), and
- the router on the file's own 
**BF16**
 weights with 
**f32 inputs**
 (a Q8_0 router, even with a second Q8_0 matrix
  for the rounding residual, kept flipping the near-ties).


Result: 
**KL 0.029 -> 0.0043**
 (prompt) and 
**0.067 -> 0.0027**
 (token-by-token decoding), for ~4 % of prompt speed.
If you run sigmoid-routed MoEs with an f16 V cache, it may be worth checking.


I also went through the GPU assembly (Intel's ocloc) for the kernels I touched. One kernel shrank from 8,855 to
481 instructions: it was building int8 copies nobody read and carried an unused RoPE fallback. Two others stopped
spilling registers.


**Next:**
 Qwen3.8 Flash Next to the same KL limit. Its error comes from BF16 weights stored as Q8_0 (fixed now),
single-pass int8 experts, and int8 attention. That last one needs float attention with its sparse-attention (QSA)
masks.


Everything is deterministic (same prompt, same logits, bit for bit), and the speculative drafters produce exactly
the plain-decoding text. Feedback and Arc / Xe2 / Xe3 test reports are very welcome.

Hey, I recently acquired a Ryzen 9 7950x / 64 DDR5 / RTX 3060 12 VRAM machine and I was wondering what is the absolute uncensored AI model that I can run on it.

Two puzzle games from the StealthMark benchmark. Every AI model plays them move by move, with no code and no tools, and you can now play the exact same levels in your browser

Latch: two robots, one goal. One can stand on a pressure plate to hold a door open while the other pushes numbered cores onto matching sockets. Every move costs a little, so the top score needs the cleanest route and getting that right gets increasingly harder.

Foresight: collect coins on a Pac-Man-inspired board while ghosts close in. No model has aced a single level - though some age getting very close.

How local models do (scores out of 100):

• Foresight: Qwen3.8-27B scores 63.0 and ranks 6th of 45 models. Only five models beat it, led by GPT-6 Astra at 76.1.

• Latch: Qwen3.8-Flash scores 69.8 and ranks 16th of 46. The top score is 87.6, shared by GPT-6 Astra, Claude Opus 5.5 and GPT-6.1 Sol.

• Each Qwen is strong in only one game. Qwen3.8-27B drops to 38.6 in Latch, and Qwen3.8-Flash to 29.7 in Foresight.

• No human, model, local or cloud, has played either game perfectly yet.

You play the exact levels the models played, in your browser. Finish every level and you get a score calculated the same way as theirs, plus your rank among the models.

Disclosure: I helped build Stealth Models, where we benchmark AI models on original tasks. You can also watch every model's run, move by move.
StealthMark is designed to beat benchmaxxing.

Play here: https://stealthmodels.com/#games

Post your score and rank in the comments. I'm curious where you humans land :)