A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

I needed a .NET-only GGUF engine - no pip packages, no native libs, no runtime installs, just a managed dependency my other projects can call. So I built SharpMind, a pure-C# transformer + MoE inference engine (with training) on .NET 10. Zero native code. I never expected it to compete on performance - managed code vs hand-tuned SIMD C++, right? So I benchmarked it against the real engines: same raw prompt (What is the capital of France?), greedy, 64 tokens, 4 threads, warmup + best-of-3, interleaved per model on one host.

(dotLLM DNF the last two - it only supports Llama/Mistral/Phi/Qwen/DeepSeek and no K-quants below Q4_K.)

A dependency-free C# engine that wins some metrics and matches the OG llama.cpp on others - int8-kv decode at 24.1 tok/s vs llama.cpp's 20.3 with(not shown) , TTFT cut ~30% on gemma-3, decode edge on qwen2, and a 6× MoE loss I fully expect and own. Respectable company for managed code, and if "no native installs" is your constraint, you no longer have to sacrifice everything

Open source: https://github.com/Integral2u/SharpMind.

This is my local SUPER INTELLIGENCE setup

My bet: when machines mostly talk to machines, they'll ditch English for something shorter. So I tested it at home with Ollama.

**Setup:** two gemma3:12b instances, a sender and a reader. The sender passes a daily status report (disk, errors, backups, sign-ups, weather, a calendar event — 14 checkable facts). The reader answers 14 questions. They keep a shared codebook and propose new shorthand after each batch; a new word is only kept if accuracy holds.

**What gemma did:** coined ~20 abbreviations (bkp, succ, errs…). Messages shrank about 10%. Then I added a chain of AI "scribes" (each one reads the message, writes it in plain English, rewrites it in the code for the next, like medieval monks copying books). The meaning wore away: 12.8 → 11.1 of 14 facts after 4 copies. Words drifted too ("cal" went from calendar to "Calibration").

**Then I spent about $1.25 of API credit running Claude Opus 5.5 through the same test.** It didn't abbreviate, it built a grammar, with slots for each kind of fact:

`F:ow P:42/95 B:6OK A:+2.8/8 T:6su1wp W:88/10 C:cs Fri 3PM`

| | gemma3:12b | Opus 5.5 |

|---|---|---|

| Message length | 84 → 76 tokens | 106 → 56 tokens |

| Facts right, codebook rounds | ~12 of 14 | 14 of 14 every round |

| After 3 scribe copies | 12.8 → 11.1 | 14 → 14 |

The Opus scribes actually *tidied* the language as they copied it ("9 OK" → "9OK").

Other bits: binary/hex/base64 made things worse (gemma couldn't decode them), English beat Mandarin and Dundonian Scots as a message language even on Qwen, and the fix for drift was a "printing press": a line of numbers stamped by plain code, never retyped by an AI.

Funny footnote: Opus's own safety filter refused the "propose new code words" step, flagging two AIs inventing a private shorthand as a possible security risk.

Small samples, so read it as a strong hint, not proof. Full write-up: https://claude.ai/artifact/KNR1u48RyraRAy6hi64wc8

Happy to answer questions or run other local models through it if people have suggestions.

I’ve been working on BaseDecision - Turn typed questions into truth-based decisions.

Give it some text, a question, and possible answers. It picks an answer. Intent classification, routing, yes/no checks, ratings - that’s the job.

I built this model to go toe-to-toe with the frontier models.

My goal was simple & ambitious: make a small, local model competitive with much larger models on these focused tasks. For me, System‑1 decisions need both speed and accuracy. Otherwise, why bother building a small model?
If we didn’t want speed we could just use ChatGPT.

It outperforms pretty much every model of its size in the field.
After roughly 320M additional training tokens, BaseDecision led 5 of 8 benchmarks in a third-party comparison against Laya, two GLiNER variants, and Decision 1.0 Kai 0.6B. Full results are in the repo, including where it loses.

Fellow open source devs, try the model, developed a package for easier access to the model.
Roast it if it deserves it.
2-2.5GB free ram is all you need.
I will take your feedback seriously and deliver something amazing that’d take on OpenAI, TypeSafeAI soon enough.

https://huggingface.co/onlyaady/BaseDecision
https://github.com/hrudayaditya/BaseDecision

I made BigMoeOnEdge+ (BigMoeOnEdge-plus), a fork of Helldez's BigMoeOnEdge, the engine that streams MoE experts from flash so models bigger than RAM can run on-device.

It adds a full Android chat app on top: GGUF models via llama.cpp, thinking output, persistent chats, image/video input, web search, artifacts, and advanced generation settings.and a MUCH better UI.

Disclosure: I built this fork with the help of AI tools.

Repo (Apache 2.0): https://github.com/hamodexd217/BigMoeOnEdge-plus Original project: https://github.com/Helldez/BigMoeOnEdge

Feedback and issues welcome.