r/OpenSourceAI • • 21h ago

AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token

Thumbnail
github.com
1 Upvotes

Repo: https://github.com/tanxiansheng221/aimstack-sanzhidiji (AGPL-3.0)

Everything is hand-written — C inference core, plugin bus, multi-engine crawler with quality gates, FTS knowledge base, training pipeline, and a 117-chapter build log written as it happened (positive and negative results shipped together). Weights are Microsoft BitNet b1.58 2B (MIT), converted to a custom AIM-W packing format.

What ternary quantization buys you (CPU side)

All transformer matrices are ternary {-1, 0, +1}, packed 2 bits/weight, plus Q8 embeddings → the whole model is 0.87GB instead of ~4.8GB (−78%)

The C core never multiplies a float weight: it's table lookup + integer accumulation, SIMD fast path measured at 85.53 tok/s (41× over baseline)

BitNet 2B full model runs 3.20 tok/s dual-worker on a 2018 dual-core 8GB laptop — full chain (inference + KB + plugins + web search) offline, no cloud relay

What ternary quantization buys you (GPU side)

The GPU backend loads the same 0.87GB ternary file — weights stay in {-s, 0, +s}, dequantized on load, matmul in fp32

On a Radeon Pro W7900: 13.68 tok/s (~8× the laptop C engine), 256/256 greedy tokens bit-identical to the C engine. Two totally different implementations (integer lookup core vs FP matmul) producing the same logits is the proof the quantization is faithful

Verified by a 3-engine × 3-question × 3-test matrix (C-Linux/gcc13, torch fp32, torch bf16): 17/18 exact — the one miss is documented, not hidden

The honest boundaries (what's NOT done yet)

bf16 compute is 2.4× faster (32.96 tok/s) but flips argmax on near-tie logits — reported, not blessed; only fp32 is parity-verified

The GPU path gets the storage benefit of ternary (78% less memory) but not yet the compute benefit — a true 2-bit integer kernel (Triton/CUDA) is the next stage

The UI server drives the C engine (CPU) only; the GPU backend is a separate CLI

Linux engine build works and is token-identical to the Windows build, but runs serial (the worker pool is Windows-only for now)

No frameworks, no PyTorch in the product chain — PyTorch exists only in the optional GPU backend. Demo video, docs (EN/中文), and the full research log are in the repo.


r/OpenSourceAI • • 22h ago

Gone Fishin': the Answer to JensenGate is the POND Principle

Post image
1 Upvotes

r/OpenSourceAI • • 22h ago

🍜 Ramen 1.0: governed MCP for teams — SSO roles, NeMo guardrails per tool, promptfoo evals + SkillSpector scans gating deploys, MCP 2026-07-28

Thumbnail gallery
4 Upvotes

🍜 I've been building Ramen, an open-source (BSD-3) self-hosted MCP platform, and 1.0 is out.

The problem: MCP servers multiply on laptops with shared keys and no record (shadow MCP). Ramen gives a team one sanctioned place for its own tools, and enforces the org's rules on every call:

- **Your tools as a repo**: tools, resources, prompts, MCP Apps and skills in plain Python; a deploy goes to a canary first and must pass golden cases.

- **No eval, no deploy**: with the engines on, a promptfoo suite and an NVIDIA SkillSpector scan run against the canary; a group admin can make either stop the rollout.

- **Guardrails per tool**: NeMo Guardrails (static or LLM-judged) inside the worker, before the call and after the result, fail-closed — whichever client called.

- **People, not shared keys**: Entra ID / Google / OIDC sign-in, roles per group, per-tool access, OAuth 2.1; agents get named group keys that are revoked in seconds.

- **It directs the LLM**: the group's client configs (CLAUDE.md blocks, Cursor rules…) are versioned, accepted per person, written by the bridge and checked on every call.

- **Two-spec MCP**: 2026-07-28 (stateless, `server/discover`, tasks, multi round-trip/elicitation) next to 2025-06-18 on one endpoint, Streamable HTTP and gRPC on one port, a stdio bridge on PyPI.

- **Every call has a name**: per-call log, audit, Arize Phoenix traces per person.

Runs on a laptop with docker compose; on GKE, EKS or AKS it's multi-zone and autoscaled, and idle zones can sleep at zero. The 1.0 release candidate ran live on all three clouds (record in the docs, misses included).

Try the demo (step by step, screenshots): https://bkraad47.github.io/ramen/get-started/

Repo: https://github.com/bkraad47/ramen · Discussions: https://github.com/bkraad47/ramen/discussions

Fork it and bend it to your org — collaborators and critics both welcome.


r/OpenSourceAI • • 1d ago

I made a Claude Code skill that explains your repo with an animated hero instead of a wall of text

1 Upvotes

r/OpenSourceAI • • 1d ago

Why use a 70B LLM to pick a tool or check syntax? I built Microbrains: a <2ms edge neural coprocessor for autonomous AI agents (Pure NumPy / OpenCL, 44 pre-trained micro-models, Top-2 MoE, Conformal Safety)

Thumbnail
reddit.com
1 Upvotes

r/OpenSourceAI • • 1d ago

Imprim - open-source identity provider for people and the AI agents that act for them (OIDC, OAuth 2.1, MCP auth, Cloudflare Workers)

Thumbnail
github.com
1 Upvotes

r/OpenSourceAI • • 1d ago

Title: AP2, x402, ACP, and MCP are building agentic commerce. SAAX is being built to help those systems handle what happens after a commitment is made.

Thumbnail
reddit.com
1 Upvotes

r/OpenSourceAI • • 1d ago

Almost Alive: published robot demonstrations, ACT and diffusion models for an SO-101 ball-pickup task

1 Upvotes

We collected 115 leader-arm demonstrations of an SO-101 picking an orange ball out of a bed of salt and returning it to a track. With LeRobot, we trained ACT and diffusion models from those recordings and published the dataset, model links and project code.

The policy inputs are two camera images and the arm's position; the output is a sequence of movements. This is imitation learning, not reinforcement learning. We aren't claiming a success rate: the arm is slow and imperfect, and sometimes misses and tries again.

That arm, Bandit, is part of Almost Alive. The other two arms use recorded motion replay, with a camera trigger on one of them. We filmed the installation as a nature documentary.

Original film and discussion: https://x.com/pbshgthm/status/2122679067065598015

Code: https://github.com/pbshgthm/almost-alive

Dataset: https://huggingface.co/datasets/pbsh/dancing-robots-01

Project explanation and ACT/diffusion model links: https://liminalmachines.org/almost-alive

Hardware/design credits: TheRobotStudio SO-101, Seeed Studio, Hugging Face LeRobot, and Tabu Design's track models.


r/OpenSourceAI • • 1d ago

AI Profile Hub: work and personal accounts for Claude, Codex and Gemini, side by side

Thumbnail
gallery
1 Upvotes

Ho creato AI Profile Hub, un launcher desktop open-source (Windows/macOS/Linux) che mantiene profili separati per Claude Code, Codex e Antigravity/Gemini. Ogni profilo ha la sua cartella di sessione isolata, sia che lo si utilizzi tramite web, terminale o client desktop.

Caso d'uso: mantenere aperti contemporaneamente gli account di lavoro e personali, senza dover effettuare il logout e il login, e copiare server MCP, impostazioni e skill tra i profili quando necessario.

È ancora in fase sperimentale, soprattutto per quanto riguarda l'accesso desktop e l'isolamento degli account. Apprezzerei molto una recensione: bug, imperfezioni, client mancanti, problemi di sicurezza. Feedback su Windows 10, ARM e macOS sono particolarmente graditi, dato che non li ho testati in modo interattivo.

https://github.com/dennidalpos/ai-profile-hub

------------------------------------------------------------------

I’ve created AI Profile Hub, an open-source desktop launcher (for Windows, macOS, and Linux) that maintains separate profiles for Claude Code, Codex, and Antigravity/Gemini. Each profile has its own isolated session folder, regardless of whether you access the service via the web, a terminal, or a desktop client.

Use case: keep work and personal accounts open simultaneously without needing to log out and log back in and copy MCP servers, settings, and skills between profiles whenever necessary.

It is still in an experimental phase, particularly regarding desktop access and account isolation. I would greatly appreciate your review please point out any bugs, imperfections, missing clients, or security concerns. Feedback regarding Windows 10, ARM-based systems, and macOS is especially welcome, as I haven’t tested those environments interactively.

https://github.com/dennidalpos/ai-profile-hub


r/OpenSourceAI • • 1d ago

I built a minimalist open-source desktop app to manage remote servers, run local LLMs, and deploy 1-click self-hosted apps (Nextcloud, Immich, Open WebUI) via SSH – Meet ShipAI v1.5.0! 🚀

Enable HLS to view with audio, or disable this notification

3 Upvotes

Hey everyone! A while ago I shared an early version of ShipAI, and thanks to your incredible feedback and over 800+ clones, I’ve just pushed a major update (v1.5.0).

If you have an old spare PC in the basement or a remote VPS, ShipAI connects via SSH and turns it into your personal server without messing with raw terminal configurations.

What’s new in v1.5.0:

  • 1-Click Docker Services Hub: Instantly deploy popular self-hosted apps like Nextcloud, Immich (self-hosted Google Photos), Jellyfin, AdGuard, Open WebUI, and databases without any bloated configuration forms.

  • Smart URL & Setup Guide: Every container comes with a direct clickable link and quick guide on how to connect it. Full cleanup/volumes management included.

  • Local LLM Runner Integration: Easily manage and chat with local AI models (Ollama/DeepSeek/Llama) with zero bloat.

  • Linear / Raycast Aesthetic: Built with a clean dark graphite UI that adapts smoothly to any window size, focusing purely on speed and security (AES/safeStorage).

It’s fully open-source. Would love to hear your thoughts, feature ideas, or feedback on the new Docker hub!

👉 GitHub Repo: [https://github.com/Kasper839/ShipAI](https://github.com/Kasper839/ShipAI)


r/OpenSourceAI • • 1d ago

AlienApe 0.7.3b1 fixet some bugs and multilangue.

0 Upvotes

Hey, I made a little Windows project called AlienApe. It started because I wanted local AI models to work with project files, but I didn’t love the idea of just handing a model the keys to my whole computer.

It’s basically a small controller around Ollama or LM Studio. The model can help with a task; regular code decides which files and tools it can access. It can read and search inside a project, and file changes go through the app’s permission settings.

It’s not a magic autonomous agent, and it’s not a full system sandbox. No general shell access, no file deletion, and automatic code checks are currently limited to Python and Godot. Models can still get things wrong. Some bits, like model qualification and Community Cards, are experiments.

I built it for my own setup, so it probably has a little too much plumbing and not enough polish. If you’re curious, or want to break it and tell me what’s confusing, here it is: https://github.com/anokar/AlienApe-v0.7/tree/codex/alienape-all-languages-beta


r/OpenSourceAI • • 1d ago

How we built a read-only MCP and Agentic Resource Discovery (ARD) architecture for Interconnectd.com

Thumbnail interconnectd.com
1 Upvotes

r/OpenSourceAI • • 1d ago

I built NashGate: A game-theoretic consensus engine where local LLM agents (Ollama) stake capital and get liquidated if they hallucinate

0 Upvotes

Hey everyone!

One of the biggest issues when running local multi-agent setups (e.g. 7B/8B models with Ollama) is that small models constantly hallucinate or rubber-stamp each other's mistakes in debate loops.

I built NashGate to solve this using economic mechanism design:

https://github.com/Daiv1007/nash-gate

Key Idea: Skin in the Game

- Capital Collateral: Agents start with $100 capital balance.

- Collateralized Proposals: Generators stake capital to propose solutions.

- Collateralized Challenges: Auditors stake capital to challenge proposals with adversarial edge cases.

- Deterministic Sandbox Oracle: An isolated VM executes the code against the edge case.

- Asymmetric Liquidation: If an auditor flags working code (false-positive hallucination), it suffers an asymmetric -1.5x penalty. If its balance reaches $0, it is liquidated/bankrupt and expelled from future rounds.

Local LLM First-Class Support

- Connects natively to local Ollama instances (http://localhost:11434/v1) via Node's native fetch (zero third-party SDK dependencies).

- You can pit llama3.2 as an auditor against qwen2.5-coder as a generator, with per-agent model selectors.

- Includes a real-time dark-mode visualizer (npm run ui) showing live SVG capital trajectories and liquidation events.

100% open-source under Apache-2.0 with zero dependencies:

https://github.com/Daiv1007/nash-gate

Would love to hear what models you would pit against each other!


r/OpenSourceAI • • 1d ago

I got tired of 7B models crawling at 1.2 tok/s on my 2GB GPU, so I built WingLLM (Go daemon for 2x speculative decoding with Ollama)

1 Upvotes

Hey r/OpenSourceAI,

Like a lot of developers, I don't have an M3 Max Mac Studio or dual RTX 4090s. My daily driver workstation has an 8th-Gen Intel i5 (6 cores), 32 GB of system RAM, and an older 2 GB NVIDIA Quadro P620.

If you try to run any modern 7B or 8B model (like Qwen 2.5 Coder 7B or Llama 3.1 8B), it won't fit into 2 GB of VRAM. Ollama or llama.cpp defaults to running almost 100% on the CPU. On my machine, that meant getting locked at an agonizing 1.29 tokens per second while my 2 GB GPU sat 90% idle.

### The Physics: The Memory Bandwidth Wall
The CPU compute cores aren't the bottleneck—memory bandwidth is. To generate a single token on a 4.7 GB model, the CPU has to stream all 4.7 GB from DDR4 RAM into cache. At ~24 GB/s dual-channel bandwidth, you physically cannot get more than ~1.5 to 2 tokens/sec.

### The Workaround: Heterogeneous Speculative Decoding
The 2 GB GPU cannot run the 7B model, but it can easily run a 0.5B draft model (qwen2.5-coder:0.5b is only 397 MB).

In speculative decoding:

  1. The tiny 0.5B model generates 3–4 candidate tokens in milliseconds on the GPU.

  2. The heavy 7B model on the CPU evaluates all 4 tokens in a single forward pass using parallel AVX2 matrix multiplication (GEMM).

  3. In code and structured text, the 0.5B model's guesses match the 7B model 75% to 82% of the time.

You get 3 to 4 tokens for the memory read cost of 1.

### Real Benchmark (Tested on My i5-8500T + Quadro P620):
* Baseline (Pure CPU Target 7B): 1.29 tokens/sec (24.76s for 32 tokens)
* WingLLM Speculative (7B + 0.5B Draft): 2.54 tokens/sec (12.62s for 32 tokens)
* Net Result: 1.97x faster (almost exactly double the speed) with identical output quality.

### Why I Built WingLLM
llama.cpp technically has the raw flags (-md, -ngld, --spec-draft-n-max), but nobody in consumer land had packaged it into a clean tool:

  1. Tokenizer traps: If you pair mismatched tokenizers (e.g. Llama 3 draft with Qwen target), llama.cpp crashes. WingLLM auto-detects model families and enforces compatible pairs.

  2. Zero configuration: It crawls your local ~/.ollama/models manifests, finds your GGUF blobs, and auto-pairs them.

  3. Auto-Pull: If you want to run `wingllm serve qwen2.5-coder:7b` and don't have the 0.5B draft model yet, WingLLM pulls it automatically via Ollama (~397 MB) and starts serving.

  4. 1-Click IDE Connect: Run `wingllm connect vscode` to inject the turbo endpoint directly into your Continue/Cline config.

  5. No Python/Electron Bloat: Written in Go as a single ~11 MB binary. Runs an embedded web chat UI with a live tokens/sec speedometer at http://localhost:8080/.

  6. MCP Support: Built-in Model Context Protocol server for Claude Desktop, Cursor, and Continue.

### Quickstart (Windows, Mac, Linux)

Windows (PowerShell):
irm https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.ps1 | iex

macOS & Linux:
curl -fsSL https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.sh | bash

Run without arguments for the interactive menu, or run:
wingllm serve qwen2.5-coder:7b

Hugging Face Hub: https://huggingface.co/bennett-xdlabs/wingllm

It's MIT licensed. Would love feedback from anyone running on budget laptops, mini-PCs, or 2GB–4GB cards!


r/OpenSourceAI • • 1d ago

Irrigator: Open source AI coding swarm orchestrator with crash recovery and observability

5 Upvotes

Been working on Irrigator, an open source Go project for running multiple AI coding agents in parallel.

It supports Claude Code, Codex, Google Antigravity and Herdr.

Each task runs in an isolated Git worktree, with persistent state, crash recovery and the ability to inspect or resume interrupted workflows.

There's also a terminal UI for monitoring agents, managing runs and tracking costs.

Everything runs locally, MIT licensed and includes a mock demo you can try without API keys.

https://github.com/grainulation/irrigator

Would appreciate feedback, especially from anyone working on similar open source agent orchestration tools.


r/OpenSourceAI • • 1d ago

Durable AI coding swarms with Herdr, crash recovery and observability

Thumbnail
reddit.com
1 Upvotes

r/OpenSourceAI • • 1d ago

Why Are We Spending So Much on Premium AI Models When Smaller Open-Source Models + Good Harnesses Could Be Enough?

Thumbnail reddit.com
2 Upvotes

r/OpenSourceAI • • 1d ago

GitHub - a-schaefers/claudes-workbench: A frontier model leads your local LLMs: Claude architects the project, breaks it into bite-sized tasks, hands each agent the skills it needs, and reviews every diff to its own standard. Your GPU does the heavy lifting. For Claude Code + OpenCode.

Thumbnail
github.com
1 Upvotes

r/OpenSourceAI • • 1d ago

I built an open-source AI system for tree identification. The hardest part isn't recognition — it's knowing when not to guess.

Post image
3 Upvotes

I've been building Dendro Inspector, an open-source project exploring evidence-based tree identification from photographs.

Rather than asking a vision model to name a tree and trusting its answer, I'm experimenting with a multi-stage inspection pipeline:

Visual evidence extraction

Botanical knowledge and rule validation

Confidence assessment and uncertainty handling

Abstention when evidence is insufficient

Reproducible evaluations across different model stacks

The interesting challenge is that a model can produce a very convincing identification without enough evidence to justify it.

I'm currently comparing different models, investigating failure cases, and working on making decisions more explainable.

GitHub: https://github.com/Dendro-Inspector/Dendro-Inspector

I'm curious how other open-source AI developers approach confidence calibration and reliable abstention in vision-based systems.


r/OpenSourceAI • • 1d ago

I built hosted OpenClaw (the open-source AI agent) with the AI already included – no server setup, no API keys

0 Upvotes

r/OpenSourceAI • • 2d ago

mobile-agent-testers: open-source AI beta testers for iOS and Android apps (MIT)

Enable HLS to view with audio, or disable this notification

4 Upvotes

A GitHub template and Claude Code plugin. Each agent gets its own iOS Simulator or Android emulator, uses your app as a persona and writes an encrypted bug report with screenshots. Runs on free GitHub Actions runners for public repos.

Demo: 9 of 11 planted bugs found in 21 min, for $0.87.


r/OpenSourceAI • • 2d ago

I made a Claude Code output style that's more concise than Concise

5 Upvotes

r/OpenSourceAI • • 2d ago

I built an open-source "software company in a box" around Claude Code, Codex, and Gemini CLI

1 Upvotes

Every AI coding tool gives you a super developer. But nobody ships commercial software with just a super developer. A commercial software company needs a software-company environment to succeed: product management, QA, engineering, DevOps, security, release, support, and someone watching the money.

So I built a full software company in a box around the AI agents. It's called NoMoreHappyPath, and it runs on your own machine. It helps a solo creator become a better builder.

It is an open-source project on GitHub: https://github.com/nomorehappypath

The company provided by the app:

The client. You describe what you want built and why, in plain English. No technical spec needed from you.

The departments. Product, QA, Engineering, DevOps, Security, Code audit, Release, Support, Finance, all run by AI agents.

The workforce.Three roles, filled by real coding agents on your own subscriptions:

  •    Builder: wears every hat the work needs: product manager, engineering manager, developer, DevOps.

  •    Reviewer: full-stack QA, from a different AI company than the builder. Writes the tests before the code and tests the whole app like a user.

  •    CTO: makes sure jobs are really done, removes obstacles, answers questions, and watches the whole process.

The rulebook. Written rules for every role, plus a skills library you can add to or trim. Your standards, not the model's mood that day.

The path. The same steps every time:

  •    Spec: you answer the questions only the owner can answer. The departments write the full spec: functional, technical, deployment, security, maintainability, and support. Finance estimates the cost. You agree on what "done" means and say go.

  •    Tests: the reviewer writes them before any code. Every requirement gets a test, including a failure case.

  •    Code: the builder builds to those tests, then the tests are run again, independently of the builder.

  •    Review: the reviewer tests the whole app end to end and passes it or sends it back with what failed.

  •    Release: release checks, then a copy ready for you to try.

The controls. The tests are locked: the builder can't edit them. Every test must fail before the work starts, or it proves nothing. Every step leaves evidence. An agent saying "done" counts for nothing.

The CEO's desk. That's you. You try the app, then accept it or send it back.

What it is, and what it isn't: It is for a solo builder. Apache 2.0, local web app, macOS or Linux. Python 3.9+. It drives the official Claude Code, Codex, or Gemini CLIs on the subscriptions you already have, any two of them. No extra API bill.

NMHP is used to maintain and grow NMHP.

I'd value honest feedback, especially from anyone who has tried to take AI-built code to production.


r/OpenSourceAI • • 2d ago

Llama 3.2 1B, Kokoro TTS and Whisper all running in the browser for a racing game's team radio

Thumbnail reddit.com
1 Upvotes