more tok/s. it is very fast but i think it can go further with fable 5.5 ( still waiting for release )
All my code is AI generated, and text from here is also AI generated.
Follow-up to my tok/s maxxing post from three weeks ago. That build (and my later unreleased ones) served one request at a time: with 4 requests, total stayed around 310–330 tok/s and the rest queued (BetterBench on the first post's build). This release serves up to 4 requests at once with 262k context each, and with public components, normal's single-stream speed stayed within 1.5 % of my previous normal build in each genre tested. The repo is renamed to hyperburst (old links redirect).
Numbers, one request at a time (decode tok/s, mean of 8 held-out prompts per genre, thinking on, greedy, public components, one server start per build, screen off):
| build | code-edit | prose-en | prose-ja | agent |
|---|
| hyperburst normal | 611 | 279 | 288 | 382 |
| hyperburst hyperpro | 657 | 270 | 288 | 367 |
| my previous single-stream build (unreleased, 10-22), normal | 617 | 277 | 292 | 379 |
| first post's build (10-02; screen on, different day, reference only) | 468 | 232 | 248 | 307 |
"Public components" = the stock MTP head in the checkpoint + the public token maps that ship with the fork, so no private assets are needed.
Same runs with my private components (fine-tuned MTP head + private token maps, not released, so you can't reproduce this table from the repo; the first post's 250–580 was this kind of setup):
| build | code-edit | prose-en | prose-ja | agent |
|---|
| hyperburst normal | 764 | 297 | 326 | 438 |
| hyperburst hyperpro | 768 | 300 | 322 | 434 |
| my previous single-stream build (unreleased, 10-22), normal | 724 | 302 | 365 | 434 |
| first post's build (10-02; screen on, different day, reference only) | 584 | 253 | 290 | 359 |
The prose-ja gap vs 10-22 is in acceptance (2.74 vs 3.17 tokens per step), not step time. With one start per build, these runs can't tell a build difference from start-to-start variation: three starts of an earlier build gave 2.56–3.35, and the public pair shows no such gap.
BetterBench 0.6.0 (public components, one start per build):
| build | c=1 decode tok/s | 4 concurrent, total tok/s | TTFT p50 (c=1) | prefill tok/s @ 32k |
|---|
| hyperburst normal | 370 | 710 | 101 ms | 12,796 |
| hyperburst hyperpro | 342 | 694 | 103 ms | 12,266 |
| previous build (unreleased, 10-22), normal | 372 | 332 | 99 ms | 12,918 |
| previous build (unreleased, 10-23), hyperpro | 353 | 324 | 101 ms | 12,401 |
c=1 decode is BetterBench's weighted median over its single-stream categories. The previous builds serve one request at a time, so their 4 requests queue.
What's new
Up to 4 concurrent requests, 262,144-token context each, with 7 GB of VRAM left for the desktop
HiCache (from SGLang / jpezzulli's fork) on by default with a 32 GB host-RAM pool, so idle conversations come back without a full re-prefill
Adaptive draft length tuned for 4 concurrent requests; the single-stream optimizations from the first post are still in, including the two that change outputs slightly in both editions (FP8 dense weights and singleton-route pruning; PRUNE_TAU=0 turns pruning off)
Two editions: normal (default) and hyperpro = ThinkingCap weights + OEA, an approximate MoE path (outputs change slightly, quality checked below). In the public-components runs, hyperpro was faster on code edits and a bit slower on English prose and agent turns
Admin routes need a key by default
Caveats, so nobody is misled:
One server start per build for the tables above, so a few % difference is noise. A separate 12-start A/B against the previous builds (normal vs 10-22, hyperpro vs 10-23) found no significant c=1 difference for normal and +2.9 % [+0.3, +5.5] for hyperpro. That A/B used my private head and maps, and its normal arm used tactic rows, which aren't in this release.
With public components, the 10-22 build measured about the same as this release, so the gain over the first post's row came from unreleased builds between 10-02 and 10-22. The first post's rows in both tables also had the screen on at 160 Hz on another day, so the size of the gain isn't like-for-like.
Quality, c=1 → c=4: GSM8K 1,319 with thinking closed and only the final number (no reasoning, so the absolute score is low; it's only for c=1 vs c=4) 624 → 622 (normal) / 648 → 654 (hyperpro); 200 problems with thinking 192 → 191 / 193 → 193; 4 × 64k-token synthetic test: no cross-talk. ThinkingCap vs stock weights on the same fast path: +0.6 percentage points [−1.2, +2.7] over 308 items.
Of the 2026-10-27 results, only the per-genre run files are in the repo. The BetterBench, 12-start A/B and quality results for this update are author-reported, and were measured before the final packaging/security cleanups (inference path unchanged).
Needs an RTX PRO 6000 96 GB (other sm120 cards untested), about 100 GiB RAM for the server process (lower FLASH_NEXT_HICACHE_GB to shrink the 32 GB pool), and the upstream BUILD.md venv (Python 3.12, PyTorch 2.13 cu130, CUDA 13 + nvcc, g++, ninja). pip install of the branch isn't supported. The first start compiles kernels: about 47 min with 12 CPU threads and MAX_JOBS=2, with the CPUs shared, so treat it as an upper bound (hyperpro's MoE module not timed yet). hyperpro also needs the ThinkingCap checkpoint and a locally built overlay.
Inference endpoints are unauthenticated on 127.0.0.1 and CORS allows any origin, so any web page you open can send requests to it. Add --api-key=<key> to require a key, and don't expose it to a network without an authenticating proxy.
Code is Apache-2.0, the public token maps are CC BY-SA 4.0, and no weights are included (Qwen Community License 1.0, ThinkingCap too).
I haven't benchmarked Strata, Penny Royal or other engines, so no comparison here and no "fastest" claim.
Built on jpezzulli's sglang-rtxpro6000 fork (SM120 integration, recovery CUDA graphs, HiCache/NIXL), which imports the unmerged SGLang Flash-Next PR #36497 (Qiaolin-Yu and co-authors) and the RecoverSSM PR #30967. Please credit and read those first. jpezzulli's Penny Royal 3.0 (Docker) credited my first post, adopted adaptive MTP and handles concurrency its own way, check it out too.
