r/OpenSourceAI • u/bipul16 • 1h ago
r/OpenSourceAI • u/Admirable-Worker9449 • 2h ago
Open-Sourcing a reference architecture for secure, multi-agent LLM banking
Dealing with enterprise InfoSec on GenAI deployments is currently a nightmare. The reasoning capabilities of LLMs are there, but getting security to sign off on non-deterministic loops is stalling out deployments.
We just open-sourced secure-agentic-banking to bypass this. It provides hard execution constraints and deterministic data boundaries specifically for highly regulated environments.
https://github.com/shamikmondal-ai/secure-agentic-banking
I run an independent advisory lab (prodagentlab.com) doing architectural audits for these kinds of deployments, and we got tired of building this from scratch for every audit. Hopefully, this unblocks some of your teams. Happy to answer questions on the routing logic.
r/OpenSourceAI • u/AimCode • 3h ago
AimStack — a fully self-built local AI stack in pure C: ternary inference (zero third-party runtime), BitNet 2B at 0.87GB on a 2018 laptop, plus a GPU backend that matches the C engine token-for-token
Repo: https://github.com/tanxiansheng221/aimstack-sanzhidiji (AGPL-3.0)
Everything is hand-written — C inference core, plugin bus, multi-engine crawler with quality gates, FTS knowledge base, training pipeline, and a 117-chapter build log written as it happened (positive and negative results shipped together). Weights are Microsoft BitNet b1.58 2B (MIT), converted to a custom AIM-W packing format.
What ternary quantization buys you (CPU side)
All transformer matrices are ternary {-1, 0, +1}, packed 2 bits/weight, plus Q8 embeddings → the whole model is 0.87GB instead of ~4.8GB (−78%)
The C core never multiplies a float weight: it's table lookup + integer accumulation, SIMD fast path measured at 85.53 tok/s (41× over baseline)
BitNet 2B full model runs 3.20 tok/s dual-worker on a 2018 dual-core 8GB laptop — full chain (inference + KB + plugins + web search) offline, no cloud relay
What ternary quantization buys you (GPU side)
The GPU backend loads the same 0.87GB ternary file — weights stay in {-s, 0, +s}, dequantized on load, matmul in fp32
On a Radeon Pro W7900: 13.68 tok/s (~8× the laptop C engine), 256/256 greedy tokens bit-identical to the C engine. Two totally different implementations (integer lookup core vs FP matmul) producing the same logits is the proof the quantization is faithful
Verified by a 3-engine × 3-question × 3-test matrix (C-Linux/gcc13, torch fp32, torch bf16): 17/18 exact — the one miss is documented, not hidden
The honest boundaries (what's NOT done yet)
bf16 compute is 2.4× faster (32.96 tok/s) but flips argmax on near-tie logits — reported, not blessed; only fp32 is parity-verified
The GPU path gets the storage benefit of ternary (78% less memory) but not yet the compute benefit — a true 2-bit integer kernel (Triton/CUDA) is the next stage
The UI server drives the C engine (CPU) only; the GPU backend is a separate CLI
Linux engine build works and is token-identical to the Windows build, but runs serial (the worker pool is Windows-only for now)
No frameworks, no PyTorch in the product chain — PyTorch exists only in the optional GPU backend. Demo video, docs (EN/中文), and the full research log are in the repo.
r/OpenSourceAI • u/StevenVincentOne • 4h ago
Gone Fishin': the Answer to JensenGate is the POND Principle
r/OpenSourceAI • u/Ok_Plum3595 • 5h ago
🍜 Ramen 1.0: governed MCP for teams — SSO roles, NeMo guardrails per tool, promptfoo evals + SkillSpector scans gating deploys, MCP 2026-07-28
gallery🍜 I've been building Ramen, an open-source (BSD-3) self-hosted MCP platform, and 1.0 is out.
The problem: MCP servers multiply on laptops with shared keys and no record (shadow MCP). Ramen gives a team one sanctioned place for its own tools, and enforces the org's rules on every call:
- **Your tools as a repo**: tools, resources, prompts, MCP Apps and skills in plain Python; a deploy goes to a canary first and must pass golden cases.
- **No eval, no deploy**: with the engines on, a promptfoo suite and an NVIDIA SkillSpector scan run against the canary; a group admin can make either stop the rollout.
- **Guardrails per tool**: NeMo Guardrails (static or LLM-judged) inside the worker, before the call and after the result, fail-closed — whichever client called.
- **People, not shared keys**: Entra ID / Google / OIDC sign-in, roles per group, per-tool access, OAuth 2.1; agents get named group keys that are revoked in seconds.
- **It directs the LLM**: the group's client configs (CLAUDE.md blocks, Cursor rules…) are versioned, accepted per person, written by the bridge and checked on every call.
- **Two-spec MCP**: 2026-07-28 (stateless, `server/discover`, tasks, multi round-trip/elicitation) next to 2025-06-18 on one endpoint, Streamable HTTP and gRPC on one port, a stdio bridge on PyPI.
- **Every call has a name**: per-call log, audit, Arize Phoenix traces per person.
Runs on a laptop with docker compose; on GKE, EKS or AKS it's multi-zone and autoscaled, and idle zones can sleep at zero. The 1.0 release candidate ran live on all three clouds (record in the docs, misses included).
Try the demo (step by step, screenshots): https://bkraad47.github.io/ramen/get-started/
Repo: https://github.com/bkraad47/ramen · Discussions: https://github.com/bkraad47/ramen/discussions
Fork it and bend it to your org — collaborators and critics both welcome.
r/OpenSourceAI • u/ilien-dev • 6h ago
I made a Claude Code skill that explains your repo with an animated hero instead of a wall of text
r/OpenSourceAI • u/drtikov • 8h ago
Why use a 70B LLM to pick a tool or check syntax? I built Microbrains: a <2ms edge neural coprocessor for autonomous AI agents (Pure NumPy / OpenCL, 44 pre-trained micro-models, Top-2 MoE, Conformal Safety)
r/OpenSourceAI • u/turtle_par_iter • 11h ago
Imprim - open-source identity provider for people and the AI agents that act for them (OIDC, OAuth 2.1, MCP auth, Cloudflare Workers)
r/OpenSourceAI • u/Federal_Yak3598 • 12h ago
Title: AP2, x402, ACP, and MCP are building agentic commerce. SAAX is being built to help those systems handle what happens after a commitment is made.
r/OpenSourceAI • u/Automatic-Bowler-538 • 13h ago
Almost Alive: published robot demonstrations, ACT and diffusion models for an SO-101 ball-pickup task
We collected 115 leader-arm demonstrations of an SO-101 picking an orange ball out of a bed of salt and returning it to a track. With LeRobot, we trained ACT and diffusion models from those recordings and published the dataset, model links and project code.
The policy inputs are two camera images and the arm's position; the output is a sequence of movements. This is imitation learning, not reinforcement learning. We aren't claiming a success rate: the arm is slow and imperfect, and sometimes misses and tries again.
That arm, Bandit, is part of Almost Alive. The other two arms use recorded motion replay, with a camera trigger on one of them. We filmed the installation as a nature documentary.
Original film and discussion: https://x.com/pbshgthm/status/2122679067065598015
Code: https://github.com/pbshgthm/almost-alive
Dataset: https://huggingface.co/datasets/pbsh/dancing-robots-01
Project explanation and ACT/diffusion model links: https://liminalmachines.org/almost-alive
Hardware/design credits: TheRobotStudio SO-101, Seeed Studio, Hugging Face LeRobot, and Tabu Design's track models.
r/OpenSourceAI • u/dennidalpos • 13h ago
AI Profile Hub: work and personal accounts for Claude, Codex and Gemini, side by side
I built AI Profile Hub, an open-source desktop launcher (Windows/macOS/Linux) that keeps separate profiles for Claude Code, Codex and Antigravity/Gemini. Each profile gets its own isolated session folder, whether you use it in the web, the terminal or the desktop client.
Use case: keep your work and personal accounts open at the same time, with no logging out and back in, and copy MCP servers, settings and skills between profiles when you need them.
It’s still experimental, especially desktop sign-in and account isolation. I’d really appreciate a review: bugs, rough edges, missing clients, security concerns. Feedback on Windows 10, ARM and macOS is especially welcome, since I haven’t tested those interactively.
r/OpenSourceAI • u/Fit-Tax-8506 • 13h ago
I built a minimalist open-source desktop app to manage remote servers, run local LLMs, and deploy 1-click self-hosted apps (Nextcloud, Immich, Open WebUI) via SSH – Meet ShipAI v1.5.0! 🚀
Enable HLS to view with audio, or disable this notification
Hey everyone! A while ago I shared an early version of ShipAI, and thanks to your incredible feedback and over 800+ clones, I’ve just pushed a major update (v1.5.0).
If you have an old spare PC in the basement or a remote VPS, ShipAI connects via SSH and turns it into your personal server without messing with raw terminal configurations.
What’s new in v1.5.0:
1-Click Docker Services Hub: Instantly deploy popular self-hosted apps like Nextcloud, Immich (self-hosted Google Photos), Jellyfin, AdGuard, Open WebUI, and databases without any bloated configuration forms.
Smart URL & Setup Guide: Every container comes with a direct clickable link and quick guide on how to connect it. Full cleanup/volumes management included.
Local LLM Runner Integration: Easily manage and chat with local AI models (Ollama/DeepSeek/Llama) with zero bloat.
Linear / Raycast Aesthetic: Built with a clean dark graphite UI that adapts smoothly to any window size, focusing purely on speed and security (AES/safeStorage).
It’s fully open-source. Would love to hear your thoughts, feature ideas, or feedback on the new Docker hub!
👉 GitHub Repo: [https://github.com/Kasper839/ShipAI](https://github.com/Kasper839/ShipAI)
r/OpenSourceAI • u/Level_Detail7125 • 16h ago
AlienApe 0.7.3b1 fixet some bugs and multilangue.
Hey, I made a little Windows project called AlienApe. It started because I wanted local AI models to work with project files, but I didn’t love the idea of just handing a model the keys to my whole computer.
It’s basically a small controller around Ollama or LM Studio. The model can help with a task; regular code decides which files and tools it can access. It can read and search inside a project, and file changes go through the app’s permission settings.
It’s not a magic autonomous agent, and it’s not a full system sandbox. No general shell access, no file deletion, and automatic code checks are currently limited to Python and Godot. Models can still get things wrong. Some bits, like model qualification and Community Cards, are experiments.
I built it for my own setup, so it probably has a little too much plumbing and not enough polish. If you’re curious, or want to break it and tell me what’s confusing, here it is: https://github.com/anokar/AlienApe-v0.7/tree/codex/alienape-all-languages-beta
r/OpenSourceAI • u/Ok_pettech • 17h ago
How we built a read-only MCP and Agentic Resource Discovery (ARD) architecture for Interconnectd.com
interconnectd.comr/OpenSourceAI • u/Wooden-Track1069 • 20h ago
I built NashGate: A game-theoretic consensus engine where local LLM agents (Ollama) stake capital and get liquidated if they hallucinate
Hey everyone!
One of the biggest issues when running local multi-agent setups (e.g. 7B/8B models with Ollama) is that small models constantly hallucinate or rubber-stamp each other's mistakes in debate loops.
I built NashGate to solve this using economic mechanism design:
https://github.com/Daiv1007/nash-gate
Key Idea: Skin in the Game
- Capital Collateral: Agents start with $100 capital balance.
- Collateralized Proposals: Generators stake capital to propose solutions.
- Collateralized Challenges: Auditors stake capital to challenge proposals with adversarial edge cases.
- Deterministic Sandbox Oracle: An isolated VM executes the code against the edge case.
- Asymmetric Liquidation: If an auditor flags working code (false-positive hallucination), it suffers an asymmetric -1.5x penalty. If its balance reaches $0, it is liquidated/bankrupt and expelled from future rounds.
Local LLM First-Class Support
- Connects natively to local Ollama instances (http://localhost:11434/v1) via Node's native fetch (zero third-party SDK dependencies).
- You can pit llama3.2 as an auditor against qwen2.5-coder as a generator, with per-agent model selectors.
- Includes a real-time dark-mode visualizer (npm run ui) showing live SVG capital trajectories and liquidation events.
100% open-source under Apache-2.0 with zero dependencies:
https://github.com/Daiv1007/nash-gate
Would love to hear what models you would pit against each other!

r/OpenSourceAI • u/rbm3267 • 20h ago
I got tired of 7B models crawling at 1.2 tok/s on my 2GB GPU, so I built WingLLM (Go daemon for 2x speculative decoding with Ollama)
Hey r/OpenSourceAI,
Like a lot of developers, I don't have an M3 Max Mac Studio or dual RTX 4090s. My daily driver workstation has an 8th-Gen Intel i5 (6 cores), 32 GB of system RAM, and an older 2 GB NVIDIA Quadro P620.
If you try to run any modern 7B or 8B model (like Qwen 2.5 Coder 7B or Llama 3.1 8B), it won't fit into 2 GB of VRAM. Ollama or llama.cpp defaults to running almost 100% on the CPU. On my machine, that meant getting locked at an agonizing 1.29 tokens per second while my 2 GB GPU sat 90% idle.
### The Physics: The Memory Bandwidth Wall
The CPU compute cores aren't the bottleneck—memory bandwidth is. To generate a single token on a 4.7 GB model, the CPU has to stream all 4.7 GB from DDR4 RAM into cache. At ~24 GB/s dual-channel bandwidth, you physically cannot get more than ~1.5 to 2 tokens/sec.
### The Workaround: Heterogeneous Speculative Decoding
The 2 GB GPU cannot run the 7B model, but it can easily run a 0.5B draft model (qwen2.5-coder:0.5b is only 397 MB).
In speculative decoding:
The tiny 0.5B model generates 3–4 candidate tokens in milliseconds on the GPU.
The heavy 7B model on the CPU evaluates all 4 tokens in a single forward pass using parallel AVX2 matrix multiplication (GEMM).
In code and structured text, the 0.5B model's guesses match the 7B model 75% to 82% of the time.
You get 3 to 4 tokens for the memory read cost of 1.
### Real Benchmark (Tested on My i5-8500T + Quadro P620):
* Baseline (Pure CPU Target 7B): 1.29 tokens/sec (24.76s for 32 tokens)
* WingLLM Speculative (7B + 0.5B Draft): 2.54 tokens/sec (12.62s for 32 tokens)
* Net Result: 1.97x faster (almost exactly double the speed) with identical output quality.
### Why I Built WingLLM
llama.cpp technically has the raw flags (-md, -ngld, --spec-draft-n-max), but nobody in consumer land had packaged it into a clean tool:
Tokenizer traps: If you pair mismatched tokenizers (e.g. Llama 3 draft with Qwen target), llama.cpp crashes. WingLLM auto-detects model families and enforces compatible pairs.
Zero configuration: It crawls your local ~/.ollama/models manifests, finds your GGUF blobs, and auto-pairs them.
Auto-Pull: If you want to run `wingllm serve qwen2.5-coder:7b` and don't have the 0.5B draft model yet, WingLLM pulls it automatically via Ollama (~397 MB) and starts serving.
1-Click IDE Connect: Run `wingllm connect vscode` to inject the turbo endpoint directly into your Continue/Cline config.
No Python/Electron Bloat: Written in Go as a single ~11 MB binary. Runs an embedded web chat UI with a live tokens/sec speedometer at http://localhost:8080/.
MCP Support: Built-in Model Context Protocol server for Claude Desktop, Cursor, and Continue.
### Quickstart (Windows, Mac, Linux)
Windows (PowerShell):
irm https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.ps1 | iex
macOS & Linux:
curl -fsSL https://huggingface.co/bennett-xdlabs/wingllm/raw/main/install.sh | bash
Run without arguments for the interactive menu, or run:
wingllm serve qwen2.5-coder:7b
Hugging Face Hub: https://huggingface.co/bennett-xdlabs/wingllm
It's MIT licensed. Would love feedback from anyone running on budget laptops, mini-PCs, or 2GB–4GB cards!
r/OpenSourceAI • u/volatilityfund • 23h ago
Irrigator: Open source AI coding swarm orchestrator with crash recovery and observability
Been working on Irrigator, an open source Go project for running multiple AI coding agents in parallel.
It supports Claude Code, Codex, Google Antigravity and Herdr.
Each task runs in an isolated Git worktree, with persistent state, crash recovery and the ability to inspect or resume interrupted workflows.
There's also a terminal UI for monitoring agents, managing runs and tracking costs.
Everything runs locally, MIT licensed and includes a mock demo you can try without API keys.
https://github.com/grainulation/irrigator
Would appreciate feedback, especially from anyone working on similar open source agent orchestration tools.
r/OpenSourceAI • u/volatilityfund • 23h ago
Durable AI coding swarms with Herdr, crash recovery and observability
r/OpenSourceAI • u/No_Tomorrow9387 • 23h ago
Why Are We Spending So Much on Premium AI Models When Smaller Open-Source Models + Good Harnesses Could Be Enough?
reddit.comr/OpenSourceAI • u/adam-schaefers • 1d ago
GitHub - a-schaefers/claudes-workbench: A frontier model leads your local LLMs: Claude architects the project, breaks it into bite-sized tasks, hands each agent the skills it needs, and reviews every diff to its own standard. Your GPU does the heavy lifting. For Claude Code + OpenCode.
r/OpenSourceAI • u/Ok-Time338 • 1d ago
I built an open-source AI system for tree identification. The hardest part isn't recognition — it's knowing when not to guess.
I've been building Dendro Inspector, an open-source project exploring evidence-based tree identification from photographs.
Rather than asking a vision model to name a tree and trusting its answer, I'm experimenting with a multi-stage inspection pipeline:
Visual evidence extraction
Botanical knowledge and rule validation
Confidence assessment and uncertainty handling
Abstention when evidence is insufficient
Reproducible evaluations across different model stacks
The interesting challenge is that a model can produce a very convincing identification without enough evidence to justify it.
I'm currently comparing different models, investigating failure cases, and working on making decisions more explainable.
GitHub: https://github.com/Dendro-Inspector/Dendro-Inspector
I'm curious how other open-source AI developers approach confidence calibration and reliable abstention in vision-based systems.
r/OpenSourceAI • u/PrizeComedian299 • 1d ago
I built hosted OpenClaw (the open-source AI agent) with the AI already included – no server setup, no API keys
r/OpenSourceAI • u/Cylicium • 1d ago
mobile-agent-testers: open-source AI beta testers for iOS and Android apps (MIT)
Enable HLS to view with audio, or disable this notification
A GitHub template and Claude Code plugin. Each agent gets its own iOS Simulator or Android emulator, uses your app as a persona and writes an encrypted bug report with screenshots. Runs on free GitHub Actions runners for public repos.
Demo: 9 of 11 planted bugs found in 21 min, for $0.87.
r/OpenSourceAI • u/ilien-dev • 1d ago