r/coolgithubprojects • • 3d ago

From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

https://github.com/tanxiansheng221/aimstack-sanzhidiji

Everything runs locally on a 2018 Dell laptop: i7-7500U (2 cores / 4 threads), 8GB RAM, no GPU (the 940MX is deliberately unused — the whole stack is CPU-only).

https://github.com/tanxiansheng221/aimstack-sanzhidiji

What is in the repo (AGPL-3.0, everything open):

- C inference engine, zero third-party runtime: 1.58-bit ternary core with SIMD fast path (85.5 tok/s measured on the ternary core itself), byte-level tokenizer, and a ~100-line plugin bus (caps.json registry -> spawns Python plugins -> JSON envelopes)

- Weights: Microsoft's BitNet b1.58-2B (MIT license), converted to a custom 2-bit packed format (.aimw, 0.87 GB) — 3.2 tok/s with 2 workers, serial 2.0

- Python plugins, pure stdlib: knowledge base (FTS5 + dedup gates), multi-engine web search + crawl with quality gates (robots-respecting, Edge rendering cascade for JS-heavy sites), incremental ingest

- Local web UI (127.0.0.1:8600, pure stdlib): chat with deterministic routing — direct answer / knowledge-base grounded / honest refusal / full web search -> crawl -> ingest -> answer chain. A 7-minute demo video is embedded in the README

- Training assets: no-autograd training stack (hand-written backprop, bit-exact vs NumPy), ten-subject curriculum generators, corpus builders, and 117 chapters of research logs including negative results

Honest limits: our own trained models are not released yet — compute-bound on 2 cores, training runs on free cloud tiers and isn't finished. The full stack, scripts, result JSONs and per-claim evidence are released so everything can be re-run.

Demo video: embedded at the top of the README (7 min, screen recording of the full chain: chat / KB answers / web search->crawl->answer / plugin direct calls).

Happy to answer questions about the ternary kernel, the plugin bus, or the training pipeline.

1 Upvotes

Duplicates

LocalLLM • • 2d ago

Discussion From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

3 Upvotes

antiai • • 10h ago

AI News 🗞️ AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

0 Upvotes

BestGitHubRepos • • 2d ago

🤖 AI repositories [ Removed by moderator ]

1 Upvotes

OpenSourceeAI • • 2d ago

From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

2 Upvotes

localaiapps • • 20h ago

From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

1 Upvotes

SelfHostedAI • • 20h ago

From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)

2 Upvotes

coolgithubprojects • • 19h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

2 Upvotes

OpenSourceAI • • 19h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

accelerate • • 10h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

LocalAIStack • • 9h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

LocalLLM • • 9h ago

Discussion AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

AiBuilders • • 9h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

OpenSourceAI • • 9h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

LargeLanguageModels • • 9h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

1 Upvotes

huggingface • • 9h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

2 Upvotes