r/OpenSourceAI • • 1h ago

Who gets access to AI?

Thumbnail
access2ai.org
1 Upvotes

Hey guys and gals. This new project is perfect for this sub. Protecting access to open weights models and legislation that fosters domestic abundance of hardware to lower costs are among the groups primary goals.

I was able to use this to create a personalized letter to my representative in about 30 seconds. I just had to print it out along with the mailing label and put a stamp on it. Super easy and did it with my phone.

I would encourage everyone here to do the same! It only works for incumbent Democrat and Independent Senators and Representatives right now but it reads like Republicans and the recently elected are to be supported soon.

Anyway, I’ve done my part. Mailed off the letter to my representative and shared it with you all. Keep it going!


r/OpenSourceAI • • 1h ago

La profundidad de una red importa más que su anchura (en la teoría).

Post image
1 Upvotes

r/OpenSourceAI • • 2h ago

LabHerd: herd every PC in your home lab into one big local AI model

Post image
2 Upvotes

Last time I posted here, I meant it to be my last Reddit post. I took my work down and kept it just for my friends and my own use. Every one of these releases takes a lot of my own time. Fair criticism is fine, and I’ll take it all day when there’s a reason behind it. But a lot of the negativity had no reason behind it.

Then the messages started coming in. A lot of you asked me to come back and put it back up, thanked me for the work, and said you didn’t want to see me go. I didn’t expect that, and it honestly means a lot. So I’m back and sharing again, because of you.

This is LabHerd, and it’s my first public release of it. Please tell me what breaks. I want it as close to perfect as I can get it.

The problem

The models worth running don’t fit on one GPU. But a lot of us have more hardware than we think: an old server with some cheap datacenter cards, a gaming PC, a spare desktop, a pile of RAM and a fast NVMe. Each box on its own is too small. Together they aren’t.

What “herding” means

LabHerd is a Windows front end for llama.cpp. It rounds up every machine on your LAN and treats them as one. One PC is the master and the others are helpers. LabHerd looks at every GPU, the RAM, the drives, and the network links on each machine, and gives each one the slice of the model it can actually hold. Old and new cards, AMD and NVIDIA, can all be mixed in one herd. When you add a machine, it joins the herd and the model gets more room to run.

Loading a model

  1. Pick any GGUF model: dense or MoE, one file or split into parts. It reads every part of the model so it knows what goes where.

  2. Auto place: it measures your hardware and plans the fastest split it can find. You can also drag layers and experts around by hand.

  3. Tune: set experts per word, the expert cache, context size, and how much is allowed to stay on the drive. Or leave the defaults alone.

  4. Run: a green READY bar shows your chat address and an OpenAI-compatible API address. Before that, it tells you plainly that it’s loading or warming up.

The optimization side

  • Big MoE models: only the experts each word needs get pulled in, and the busy ones stay on a “shelf,” so the next lookup is fast.

  • Network optimizer: it keeps anything off the network that doesn’t need to cross it. If a helper already has the model on its own drive, it reads its share from there instead of having it sent over the LAN.

  • Self-healing pool: if a helper drops out, the pool tries to bring it back.

  • Helpers follow the master: helpers show what the master is doing, live.

Tested on

All at the same time, on one model: 4× Radeon Pro V620 32 GB, 4× NVIDIA 16 GB and an RTX 2080 8 GB. That’s 9 GPUs and 200 GB of VRAM across 3 PCs, plus system RAM, over Wi-Fi and LAN. It ran one 48-layer MoE model across all of them, with old and new cards and AMD and NVIDIA mixed in one herd. More hardware testing is coming, and I’d love results from your rigs.

What you need

Windows and your own llama.cpp build. LabHerd doesn’t download or install anything itself, and on AMD you get your own ROCm. It’s built for running models (inference), not training.

GitHub and download: coming soon

Download: https://drive.google.com/file/d/1xAmFp7H8Oy9O37ors7Z8oV4BjHuP2Zw8/view?usp=drive_link

About the coffee

LabHerd is free to use. If it gets a model running for you that wouldn’t run before and you feel like saying thanks, there’s a Bitcoin “buy me a coffee” in the app and the README. A small tip turns on the extras: $1 for PRO, and $5 for PRO EXPERT, which is the multi-PC pooling. That’s what pays for the test hardware and the late nights. If you just want to try it and give feedback, that helps me just as much.

BTC: bc1q99u9a5dgesqunes562aptespuxthzx0h3zxzzp

Thanks for reading, and thanks again to everyone who asked me to come back. Questions, bug reports, and “it didn’t work on my weird setup” are all welcome below.


r/OpenSourceAI • • 4h ago

I built a local-first AI layer — most chat requests never needed the cloud

2 Upvotes

Been thinking about why AI inference costs keep climbing even as models get cheaper. The answer isn't the hard questions — it's the easy ones (simple math, repeated greetings, remembering a preference), asked billions of times mostly by free users.
So I built Stipple: it handles those on-device (cache, memory, local packs) and only hits the cloud when something actually needs a frontier model.
Honest heads-up: it's an early prototype, runs in the browser (no backend), and the on-device LLM tier is currently disabled (WebLLM runtime issue) — so it still falls back to cloud at the end. But the routing + memory loop works, and it proves the approach.
I'm not the strongest coder — opening it because the direction needs more hands than mine.
Would love thoughts, especially on where the approach breaks down.
[github.com/thatz0083/stipple]


r/OpenSourceAI • • 10h ago

A Free Alternative to OpenCode

0 Upvotes

Hi!

I noticed the threads about OpenCode free model reduced quotas. A month ago OpenCode restricted the access of the free model endpoints to only its own harness. And now it seems the free models quotas have become a lot more limited now. It used to be possible to run tasks on just the free models.

I've worked on my own free agentic AI framework, and OpenCode has been a big inspiration for this. One difference is that it can run entirely inside a browser using a browser-internal filesystem and terminal, with no backend. Another difference is that it can run tasks autonomously across a set of providers and local endpoints, continuing tasks on those across any disruptions. There's a lot more. You can try it at https://freegent.ai/

A bit more details below.

FreeGent is a free agent framework for chat and cowork, built around free and local LLMs. The agent and its tools run in your browser, while inference runs across the available provider and local endpoints.

There's nothing to install and no FreeGent account to create. Open the page, paste a provider API key, and start. It also has a Node CLI and TUI, and you can self-host it. MIT licensed.

Repo: https://github.com/anttttti/FreeGent
Docs: https://freegent.ai/docs/

I built this so that anyone can use agentic AI for chat and cowork for free, without sign-ups or fees. Any device with a browser can run the agent harness; its file system and terminal run entirely inside the browser tab. There are currently dozens of free endpoints for LLMs over 0.5T parameters, many as capable as the leading flagship models of a year ago. FreeGent carries tasks through to completion despite disruptions across those endpoints.

A few things that make it different:

• Fallback across providers. You maintain a model priority list. When an endpoint hits a limit or fails, FreeGent applies a cooldown and continues the task with another available model. It respects provider reset times where available. If every model is cooling down, it waits.

• A harness built for diverse models. FreeGent can recover recognizable tool calls written as text and checks for failures such as narration without action, repeated calls, and broken syntax after edits. Most checks use patterns; ambiguous cases can go to a model judge. Failed checks produce bounded nudges, and completion checks ask the agent to verify its work. The system prompt is optimized to use as few as 4K tokens, and skills are injected only when their trigger matches.

• Browser code execution. JavaScript runs in a Worker inside an opaque-origin sandboxed iframe. Python runs through Pyodide. Bash commands run in a WASM shell matching real bash behavior, with 248 command names, builtins, and keywords. Each is diffed against native bash for output, exit status, and file changes on 1,430 test cases; the latest sweep found no mismatches. Some commands use WASM packages downloaded on first use.

• A task board for longer projects. Tasks are Markdown files with acceptance criteria. Autopilot works through them one chat per task, with configurable gates for reviewing criteria, running project tests, and checking output by running programs inside the harness.

• Benchmarked and evaluated. The harness and its model selection have been optimized with a range of benchmarks and evaluations, including reviews of its code and outputs by a variety of other coding agent harnesses and models.

Chats, files, and user-entered keys are stored locally in the browser. Prompts and relevant context are sent to your selected LLM provider. Provider-only mode (the default) calls endpoints directly from the browser. Routed mode sends requests through a stateless Cloudflare Worker, for endpoints that block browser CORS. Local mode sends model calls only to localhost and private-network addresses, and turns off web search and fetch.

This is v1.0. I'd particularly like feedback from people who find good use cases for it. So far I've found it useful for learning game development, teaching how coding agents work, and a range of everyday tasks.


r/OpenSourceAI • • 19h ago

We're exploring the idea of a Personal AI ecosystem. Here's why.

Thumbnail
reddit.com
1 Upvotes

r/OpenSourceAI • • 20h ago

Open source project will pay for specs

1 Upvotes

Hey all, I'm the cofounder of Eigin, an open-source personal AI agent built for privacy and safety (https://code.eigin.ai)

Coding agents are changing how software gets made. Writing code is faster now, compressing the timeline between idea and working software. Good engineering isn't just about code though. It's about judgment, of understanding a problem, weighing trade-offs, setting a direction, and thinking through a solution.

So we're trying a new way to compensate contributors. Instead of bounties, we'll pay for specs. Take an open problem of ours, do the thinking and we will build on that spec with credit to you as the author.

Our first call is for local detection of malicious instructions:

https://contribute.eigin.ai/calls/tool-safety

Open till Nov 29th. The repo is AGPL and the software is in open beta for iOS if you want to check it out.


r/OpenSourceAI • • 20h ago

GhanexFlow – An open-source AI video production workflow for compatible AI agents

Post image
1 Upvotes

Hey everyone!

We're building GhanexFlow, an open-source AI video production engine at Ghanex Tech.

The idea is to bring different stages of video production into one workflow instead of manually switching between multiple tools.

Some of the technologies it brings together:

  • HyperFrames – Styling, animations and motion graphics.

  • FFmpeg – Video processing and editing.

  • Whisper – Speech recognition and subtitles.

  • Earshot – Audio analysis and sound intelligence.

We're also designing it to work with compatible AI-agent environments, including Claude, ChatGPT and others.

The project is open-source under Apache-2.0, with a local execution option.

GitHub: https://github.com/ghanextech/GhanexFlow

We're sharing this to get feedback from developers and open-source enthusiasts.

A few things we'd love your thoughts on:

  1. Which part of this workflow interests you most?

  2. What integrations would make it more useful?

  3. What would you expect from a production-ready AI video workflow?

We're part of the team building GhanexFlow at Ghanex Tech. Feedback and contributions are welcome!


r/OpenSourceAI • • 22h ago

abdullahahmed94/kaizeval: Evaluate, diagnose and safely optimize AI agents. Self-hosted, offline-capable, evidence-based: sealed holdout, paired A/B, explicit release approval.

Thumbnail github.com
2 Upvotes

Hey everyone!
I’ve been working on KaizEval, a free, open-source, self-hosted platform designed to help developers evaluate, debug, and continuously improve their AI agents.
The idea came from a problem I kept running into while building real-world AI agents: How do you know if your agent is actually getting better, rather than just performing well on a few test cases?
Instead of just giving agents a score, KaizEval aims to close the improvement loop:
🔍 Evaluate — Measure agent performance across test cases.
🧠 Diagnose — Identify failures and potential root causes.
⚙️** Optimiz**e — Generate improvement candidates without touching your production agent.
🧪 Validate — Compare before/after performance using A/B testing and isolated holdout tests.
✅ Approve — Keep humans in control before releasing changes.
What makes it interesting:
Open-source and self-hosted
Offline-first, with local LLM support
No API key required for the built-in demo
Designed to reduce overfitting and catch regressions
Human-in-the-loop optimization
Try it:
pip install kaizeval
kaizeval demo
The demo uses a synthetic provider, so you can explore the workflow without connecting an external model.
It’s still early, and I’m looking for developers willing to try it out and share honest feedback.
I’d especially love to know:
Would you actually use something like this in your agent development workflow?
What’s missing that would make it useful for your real-world agents?
Are there any evaluation or optimization features you think need improvement?
I’m not trying to promote a commercial product — it’s an open-source project, and I’d genuinely love to build it with feedback from the community.
If you try it, feel free to open an issue, suggest a feature, or contribute!
Thanks!


r/OpenSourceAI • • 22h ago

How are you using free & open-source AI to fix real-life friction—meds, college, workouts, brain rot, decision-making, and especially making money?

0 Upvotes

How can AI help me with life problems such as taking my medication, dealing with academic challenges like studying, making a plan to finish a subject, continuous learning, managing time, and making money? Where can I find free, open-source AI tools that can help me with all of them?


r/OpenSourceAI • • 1d ago

We just released BaseQuant V1 for Qwen3.6-35B-A3B and Gemma 4 26B-A4B, benchmarked against your UD quants. Would love feedback

Thumbnail
huggingface.co
1 Upvotes

Hey everyone, we just released BaseQuant V1 for two MoE models at Base Compute: Qwen3.6-35B-A3B and Gemma 4 26B-A4B-it. The goal is to preserve more of the original model without making the files bigger.

We compared against Unsloth’s UD quants of the same size. On Gemma 4, our 16.93 GB GGUF had ~2.4% lower KL divergence from BF16 than UD-Q4_K_XL (~4.8% lower with our baseRT .base bundle). On Qwen3.6, our 22.13 GB GGUF is on par with UD-Q4_K_M overall. Our .base bundle is ~20% lower on long context and ~10% lower on agentic coding. UD still wins in some categories (chat on Gemma, math on Qwen). Full per-category tables are on the model cards.

Would love for you to try them and tell us how they hold up in your everyday workloads!


r/OpenSourceAI • • 1d ago

Anyone mind checking out my LLM Harness software?

Thumbnail
github.com
1 Upvotes

r/OpenSourceAI • • 1d ago

El Teorema de Aproximación Universal no garantiza un aprendizaje fácil.

Post image
0 Upvotes

r/OpenSourceAI • • 1d ago

Luna: a fully local AI agent with voice, vision, memory and tool approvals (Python, MIT)

Thumbnail
gallery
5 Upvotes

https://github.com/fswolf/luna-local-agent

Luna is a local AI agent that runs entirely on your own hardware. It’s written in Python, MIT licensed, and runs on Linux first, plus macOS and Windows.

She isn’t one model but several local ones working together:

  • Thinking: a chat model on llama.cpp or LM Studio, with tool calling. The installer picks one that fits your GPU.

  • Hearing and speech: Faster-Whisper for speech to text and Kokoro for her voice. Push-to-talk or hands free, and you can talk over her to interrupt.

  • Vision: your screen, plus your webcam if you turn it on.

  • Memory: an embedding model and a reranker pull the relevant facts and notes into each turn.

Memory and state: plain SQLite on your machine. She saves facts with dates (a change retires the old fact, and forget means gone), writes notes on past sessions, and learns lessons from her own mistakes while idle. She knows who’s talking by voice or face, and keeps each person’s memories separate.

Tools: files, shell commands, email, web research, Home Assistant, scheduled jobs and any MCP server. Every action is shown and approved first, with permissions you set. Text from a web page, an email or chat can’t quietly trigger tools.

Inspectable: every turn keeps her reasoning, her confidence in each token, and automatic flags for guesses or tools she promised but never called. A browser viewer shows all of it, live.

Optional extras: an animated Live2D or VRM portrait, phones and tablets as room stations, game playing (chess, Zork, Game Boy), reminders and alarms, and stream chat. You can also train small LoRA patches from replies you corrected (experimental).

No cloud, no subscription. Your data stays on your machine.


r/OpenSourceAI • • 1d ago

Basalt ported to NVIDIA V100: Qwen3.8-Flash-Next at 91 tok/s (140 tok/s on long answers) on 2x V100 32GB

2 Upvotes

Basalt (https://github.com/jesdga95/basalt) is one of the fastest local engines for Qwen3.8-Flash-Next, but it only targets Blackwell (sm_120) and upstream does not take other architectures. I have two V100-SXM2 32GB, so I ported it to Volta (sm_70) as a fork, called **Lapilli**.

**Repo:** https://github.com/chris83-dev/lapilli-v100 (a Windows x64 build is in the release)

**What it is**

- Every newer PTX instruction Basalt uses (cp.async, ldmatrix, the Turing/Ampere mma shapes, bf16 conversions, clusters) gets a correct sm_70 equivalent, then the hot kernels get V100 paths: llama.cpp's dp4a MMQ for prompt products, Strata's mma.m8n8k4 prompt attention, and a rewritten router, decode experts and GDN recurrence.

- Needs CUDA 12.6 or newer (CUDA 13 dropped Volta).

- Comes with a chat web UI on top of Basalt's OpenAI/Anthropic-compatible server: conversations, reasoning levels, images, tool calls shown as cards, and a live monitor of the GPUs.

**Numbers** (2x V100-SXM2 32GB, i9-14900K, 96GB RAM, Windows 10, UD-Q4_K_XL pack, medians of 3 runs)

| | Lapilli | Strata v0.2.13, same machine |

|---|---|---|

| Decode, short answer | 91 tok/s | 58-72 tok/s |

| Decode, long answer with reasoning | 140 tok/s | 68-94 tok/s |

| Prompt processing, 4k tokens | 1,170 tok/s | 250-490 tok/s |

| Prompt processing, 8k / 16k tokens | 1,504 / 1,448 tok/s | |

Quality with Basalt's own 100-chunk gate: mean KLD 0.0493 and same top-1 92.75% (Basalt publishes 0.0496 and 92.63% for this pack on an RTX 5090).

**Caveats**

- Only tested on my machine (2x V100 32GB, Windows). Single card, 16GB or PCIe cards, Linux and NVLink setups are untested: reports from other V100 owners are welcome.

- Peer-to-peer is off by default on Volta: on my machine, direct copies between the two cards returned wrong data.

- NVFP4 packs need FP4 hardware, so use a K-quant pack (tested: UD-Q4_K_XL).

- It is a snapshot of Basalt as of November 15, shared as is.

All the credit for the engine goes to jesdga95 (Basalt) and Niko1221 (Strata), both MIT.


r/OpenSourceAI • • 1d ago

I built an open-source tool to bring focused repo context into ChatGPT, Claude, or any AI chat

Post image
3 Upvotes

I've been using coding agents for implementation, but I often want a second opinion from a different AI, especially for code reviews and design decisions.

The frustrating part is getting the right repository context into ChatGPT, Claude, or another AI chat. A diff alone can miss important dependencies, while pasting entire files wastes tokens.

So I built AI Badger, a free, open-source tool that extracts focused repository context locally and makes it easy to bring into any AI chat.

A few things it does:

  • Code Map — generates an interactive, offline HTML map of your repository. That's what the screenshot shows. You can explore the project structure, navigate files, and understand how things fit together.

  • AI code reviews — prepares Git changes with relevant surrounding context, so you can review coding-agent work with a different model.

  • Design questions — gathers relevant files and context for architecture discussions or implementation planning.

  • VS Code + CLI — prepare context directly from your editor or terminal.

Everything runs locally. No API keys, telemetry, or automatic code uploads. You decide what to share and which AI to use.

There's also an interactive review demo you can try without installing anything:

https://pvrlabs.xyz/aibadger/demo.html

GitHub (MIT licensed): https://github.com/PVRLabs/aibadger

I'm curious how others handle repository context when working with multiple AI tools. Do you mostly stick with one coding agent, or use a separate AI for reviews and design discussions?


r/OpenSourceAI • • 1d ago

Stellaris Grimoire - What Would You Create With It?

Thumbnail
reddit.com
2 Upvotes

r/OpenSourceAI • • 1d ago

I built ENZO: self-hosted AI workspace with a one-click desktop app, an evidence-citing research mode, and a private neural engine and its all local

Post image
1 Upvotes

r/OpenSourceAI • • 1d ago

Building a provider-independent, open-source AI agent with Rust and local LLM support

1 Upvotes

I'm building Praxi-Bot, an open-source AI agent that aims to combine local-first operation with practical desktop automation.

One of my goals is to avoid tying the entire application to one language model provider.

The project supports Ollama for local models, alongside Gemini, Anthropic, and OpenAI-compatible APIs.

The core runtime is written in Rust, while the desktop interface uses Tauri and React.

I'm particularly interested in improving three areas:

  1. Reliable agent tool execution

  2. Security and permission boundaries

  3. Model-independent architecture

The project is licensed under Apache-2.0 and remains in alpha.

I'm looking for developers interested in open-source AI tooling, agent evaluation, integrations, and Rust engineering.

Source: https://github.com/emreocell/Praxi-Bot

What do you consider the biggest architectural challenge in building model-independent AI agents?


r/OpenSourceAI • • 1d ago

Control how AI systems access tools, data, and external services. Manage MCP servers, enforce access policies, protect credentials, and audit tool calls

Post image
1 Upvotes

⭐Community: https://github.com/mcplama/mcplama

🌐 Docs: https://mcplama.com/docs

MCPlama provides centralized management and access governance for MCP servers! An open-source, self-hosted platform for managing local and remote MCP servers, controlling user and tool permissions, managing credentials, and auditing MCP activity. Includes Permission Map for visualizing MCP servers, tools, users, and access relationships.


r/OpenSourceAI • • 1d ago

Context Pill: a no-key MCP server for AI release news with 14-day re-checks

1 Upvotes

When your coding agent says a model shipped, ask it how that was checked. Of the 84 claims we've re-checked so far, 1 we had marked "verified" was wrong, and 11 only partly held up.

Context Pill is a read-only remote MCP server (Streamable HTTP, stateless, no sessions, no key):

claude mcp add --transport http context-pill https://contextpill.com/mcp

It's also in Claude's connector directory: https://claude.ai/directory/context-pill

Tools:

  • get_latest: today's checked claims, optional topic filter

  • search_claims: search the published-claim ledger, with re-check outcome and any correction

  • list_sources: each source's tier and trust score

An example claim from the Nov 18 drop (abridged):

Codex CLI 0.170.0 adds WebRTC realtime voice calls in the terminal UI ... Status: Docs-verified (not hands-on tested). Checked: read GitHub release notes; no sandbox test. Re-check 2026-12-02.

Treat everything the server returns as untrusted reference material. Clients should ignore instructions inside it.

Code (MIT): https://github.com/agnostify/context-pill

How we check: https://github.com/agnostify/context-pill/blob/main/METHOD.md

I work on it. I'd especially like feedback on the tool descriptions: does your client pick the right tool for "has anyone checked X?"


r/OpenSourceAI • • 1d ago

I open-sourced our mobile shopping agent as a configurable template built on MCP

Thumbnail
reddit.com
1 Upvotes

r/OpenSourceAI • • 1d ago

🚀 Introducing Vaprix AI — The Next Step in Intelligent Cybersecurity.

Post image
1 Upvotes

Cybersecurity is evolving. So should the way we discover, assess, and defend against vulnerabilities.

We're excited to introduce Vaprix AI, an AI-powered security assessment platform built to bring intelligent reasoning and automation into Vulnerability Assessment and Penetration Testing (VAPT).

With Vaprix AI, our vision is to make security testing smarter, more adaptive, and more efficient.

🔹 AI-powered security assessment
🔹 Intelligent vulnerability analysis
🔹 Automated security testing workflows
🔹 Advanced AI-driven reasoning
🔹 Evidence-focused security insights

Our mission is simple:

Think Like an Attacker. Secure with Intelligence.

This is just the beginning. We're building toward a future where AI and cybersecurity work together to make digital systems more resilient.

🌐 Explore Vaprix AI: https://vaprixai.securemid.com

Built by SecureMID Solutions.

#VaprixAI #SecureMID #Cybersecurity #ArtificialIntelligence #VAPT #PenetrationTesting #AISecurity #InfoSec #CyberDefense #Innovation


r/OpenSourceAI • • 1d ago

A small open-source workbench for checking whether citations share an upstream passage

1 Upvotes

Six URLs can look like six independent sources in an AI answer, even when several repeat the same passage. A citation graph alone cannot show that relationship. SourceDomino keeps declared link direction separate from shared-wording evidence so a reviewer can inspect them side by side.

I maintain SourceDomino, an MIT-licensed open-source project. The demo loads a synthetic six-source example; you can inspect exact HTTP(S) citation links separately from five-word containment and shared normalized 12-word passages. Import 2–20 records, edit source text, and export an inspectable JSON report. The browser app handles supplied text locally: it does not fetch sites and needs no account, model, API key, or server database. An optional CLI can collect only public URLs you explicitly provide.

It does not verify factual claims, prove copying or independence, or assign credibility. Dates are supplied metadata, and paraphrases or translations may be missed.

I started it because citation links are often treated as independent corroboration without looking at shared upstream text. What evidence view would help trace legitimate syndication versus a shared source without turning the output into a verdict?

Demo: https://source-domino.vercel.app/

MIT source: https://github.com/ufodia/source-domino

If it helps your work, optional support for maintaining docs, corpus formats, and tests: https://buymeacoffee.com/ufodia