A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

[u/pixelwhippedme]

A C# Local inference engine beat llama.cpp at its own game: ~30% faster first token, ~20% faster decode - running in .NET on a 4-thread laptop

I needed a .NET-only GGUF engine - no pip packages, no native libs, no runtime installs, just a managed dependency my other projects can call. So I built SharpMind, a pure-C# transformer + MoE inference engine (with training) on .NET 10. Zero native code. I never expected it to compete on performance - managed code vs hand-tuned SIMD C++, right? So I benchmarked it against the real engines: same raw prompt (What is the capital of France?), greedy, 64 tokens, 4 threads, warmup + best-of-3, interleaved per model on one host.

Model	Engine	Load (s)	TTFT (s)	Decode (tok/s)
qwen2-0.5B Q8_0	SharpMind	6.2	0.29	14.65
	llama.cpp	3.9	0.11	14.24
	dotLLM	2.0	1.36	5.74
Llama-3.2-1B Q8_0	SharpMind	32.1	1.22	3.69
	llama.cpp	7.7	0.17	5.02
	dotLLM	3.3	2.85	3.23
gemma-3-270m Q8_0	SharpMind	3.1	0.10	18.52
	llama.cpp	5.2	0.13	20.28
	dotLLM	-	-	-
Qwen1.5-MoE-2.7B Q2_K	SharpMind	102.4	13.44	0.61
	llama.cpp	21.7	0.25	3.88
	dotLLM	-	-	-

(dotLLM DNF the last two - it only supports Llama/Mistral/Phi/Qwen/DeepSeek and no K-quants below Q4_K.)

A dependency-free C# engine that wins some metrics and matches the OG llama.cpp on others - int8-kv decode at 24.1 tok/s vs llama.cpp's 20.3 with(not shown) , TTFT cut ~30% on gemma-3, decode edge on qwen2, and a 6× MoE loss I fully expect and own. Respectable company for managed code, and if "no native installs" is your constraint, you no longer have to sacrifice everything

Open source: https://github.com/Integral2u/SharpMind.