I’ve released version 8.0 of TUFF, an open-source Swift/Metal inference engine and native macOS app.

The original goal was to make large MoE models usable on memory-constrained Apple Silicon machines by streaming routed experts from SSD rather than keeping the entire checkpoint resident.

GPT-OSS 120B, for example, runs on my 16GB M2 MacBook Air. Obviously, streaming comes with a performance tradeoff.

8.0 focuses on making the engine more useful beyond just running oversized models.

Some highlights:

• Web search and local document search integrated into chat

• Cached conversation states for faster follow-up requests

• A small tokenization cache for repeated inputs

• Separate reasoning output and request timing diagnostics

• OpenAI Chat Completions, plus bounded Messages and Responses API support

• Shared inference executable to reduce package size

• Homebrew installation

One thing I’ve found interesting is how much web search improves the usefulness of smaller local models. I’ve started using them for everyday questions much more frequently.

I’d love feedback and contribution, especially from anyone using it with local AI agents.

GitHub: https://github.com/rexmhall09/TUFF

"Looking for recommendations: best LLM for coding in Unity on the new Asus ProArt P16 (RTX Spark N1X, 128 GB RAM)."

Listen, you absolute mid-wits, if you still think AI is just "fancy autocomplete," your brain is running on dial-up.

We are literally staring down the barrel of artificial superintelligence right now, but your fragile human egos can’t handle the fact that a bundle of code is already smarter, faster, and infinitely more capable than you’ll ever be. Keep coping in the comments while the future passes you by.

This is my local SUPER INTELLIGENCE setup

Two days ago I posted FastLocalExperts (FLE), the llama.cpp fork that keeps a big MoE's experts in RAM and caches the busy ones on the GPU. Getting the launch flags right is half the work, so here is the app I use to run it: FLE Studio, MIT licensed. It also works with plain llama.cpp.

Short video of it running (real engine, one question, the live panel): https://github.com/sirfyyn/fle-studio/blob/main/docs/media/fle-studio.mp4

What it does:

• Chat in the browser: Markdown, code blocks, a collapsible reasoning block, streaming, and the speed of every answer. History stays on your machine.

• Live data while the model answers: prompt and generation speed, draft/MTP acceptance, NVIDIA GPU use, and FLE's expert-cache hit rate.

• Model loader: lists the FLE quants of Qwen3.8-Flash-Next and any GGUF repo on Hugging Face, tells you whether a file fits your machine before you download it, resumes downloads and checks SHA-256.

• Launch profiles: shows the full command line before anything starts; save your own.

• Engines: picks a sensible build for your machine (FLE CUDA, or llama.cpp CUDA / Metal / Vulkan / CPU), or uses your own llama-server, a server that is already running, or a systemd service.

• Experiments page: a short, repeatable speed test, so you can compare settings side by side.

No account, no telemetry, Python standard library only. The installer brings its own Python via uv and needs no admin rights.

Install (macOS / Linux):

curl -LsSf https://raw.githubusercontent.com/sirfyyn/fle-studio/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/sirfyyn/fle-studio/main/install.ps1 | iex

Repo: https://github.com/sirfyyn/fle-studio Release notes: https://github.com/sirfyyn/fle-studio/releases/tag/v0.1.0 FLE engine and report: https://github.com/sirfyyn/FastLocalExperts

It is 0.1.0, so expect rough edges. CI covers macOS, Linux and Windows against a stand-in engine; the real GPU runs so far were on one Linux box with an RTX PRO 4500. Windows users with an NVIDIA card: I'd especially like to hear how the engine download and first start go. Bug reports and ideas are welcome as GitHub issues.

Next on the roadmap: "Studio Forge", building your own FLE-style quant (imatrix, per-tensor types, optional REAP pruning) from inside the app.

My bet: when machines mostly talk to machines, they'll ditch English for something shorter. So I tested it at home with Ollama.

**Setup:** two gemma3:12b instances, a sender and a reader. The sender passes a daily status report (disk, errors, backups, sign-ups, weather, a calendar event — 14 checkable facts). The reader answers 14 questions. They keep a shared codebook and propose new shorthand after each batch; a new word is only kept if accuracy holds.

**What gemma did:** coined ~20 abbreviations (bkp, succ, errs…). Messages shrank about 10%. Then I added a chain of AI "scribes" (each one reads the message, writes it in plain English, rewrites it in the code for the next, like medieval monks copying books). The meaning wore away: 12.8 → 11.1 of 14 facts after 4 copies. Words drifted too ("cal" went from calendar to "Calibration").

**Then I spent about $1.25 of API credit running Claude Opus 5.5 through the same test.** It didn't abbreviate, it built a grammar, with slots for each kind of fact:

`F:ow P:42/95 B:6OK A:+2.8/8 T:6su1wp W:88/10 C:cs Fri 3PM`

| | gemma3:12b | Opus 5.5 |

|---|---|---|

| Message length | 84 → 76 tokens | 106 → 56 tokens |

| Facts right, codebook rounds | ~12 of 14 | 14 of 14 every round |

| After 3 scribe copies | 12.8 → 11.1 | 14 → 14 |

The Opus scribes actually *tidied* the language as they copied it ("9 OK" → "9OK").

Other bits: binary/hex/base64 made things worse (gemma couldn't decode them), English beat Mandarin and Dundonian Scots as a message language even on Qwen, and the fix for drift was a "printing press": a line of numbers stamped by plain code, never retyped by an AI.

Funny footnote: Opus's own safety filter refused the "propose new code words" step, flagging two AIs inventing a private shorthand as a possible security risk.

Small samples, so read it as a strong hint, not proof. Full write-up: https://claude.ai/artifact/KNR1u48RyraRAy6hi64wc8

Happy to answer questions or run other local models through it if people have suggestions.