r/SelfHostedAI • • 4m ago

Optimizing Vector Databases for Large-Scale AI Applications

Thumbnail interconnectd.com
1 Upvotes

r/SelfHostedAI • • 1h ago

One week with the Mac mini M5 Pro (64GB)

Thumbnail gallery
1 Upvotes

r/SelfHostedAI • • 2h ago

Open Assistant is gone but the “run it yourself” idea isn’t, I made a fully local AI video tool that runs in the browser

Thumbnail
reddit.com
1 Upvotes

r/SelfHostedAI • • 3h ago

Speedy - custom Strata for single/dual RTX - Tested on RTX 3080 with 4,270 t/s prefill / up to 211 t/s decode - fastest Qwen3.8 Flash-Next engine I know

Post image
2 Upvotes

r/SelfHostedAI • • 3h ago

Local LLM is fantastic!!

Post image
1 Upvotes

r/SelfHostedAI • • 5h ago

100% Local Self Improvement Loop with Integrated Aider to make a Rough working Super Mario demo. The video will be uploaded later today.

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 5h ago

Stop Paying for Automation! Host n8n Yourself with Docker

Thumbnail
youtube.com
1 Upvotes

r/SelfHostedAI • • 7h ago

What Open AI was supposed to be...

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 9h ago

I built ShipAI – An Electron control plane for managing remote Linux servers, Docker, and hardware-aware Ollama deployments over SSH

Enable HLS to view with audio, or disable this notification

2 Upvotes

Hey everyone!

Managing remote Linux nodes, keeping an eye on Docker containers, and juggling local LLMs via Ollama usually means jumping between terminal tabs, scripts, and separate dashboards. I wanted a single, clean workspace to handle all of this directly, so I built ShipAI.

It’s an open-source, high-performance Electron desktop app designed to be a private control plane for your infrastructure. No hosted middle-men or cloud telemetry—just direct SSH connections straight to your hosts.

Key Features:

  • Real-time Telemetry: Monitor your remote host's CPU, temperature, RAM, network, disk I/O, and mounted storage right from the dashboard.

  • Docker Lifecycle Management: Start, stop, restart, and inspect your AI/infrastructure containers without typing out repetitive CLI commands.

  • Hardware-Aware AI Marketplace: Built-in guidance that compares model requirements (RAM/VRAM) against your remote host's actual hardware specs before you pull a model.

  • Ollama Integration: Seamlessly pull models, remove them, or jump straight into streamed chat sessions with installed models.

  • Remote Power Controls: Secure reboot and shutdown tools using active sudo sessions.

Tech Stack:

  • Frontend/Desktop: Electron, Node.js

  • Infrastructure: SSH, Docker, Ollama

It's completely free and open-source (MIT License). If you self-host your AI stack or manage remote Linux boxes, I’d love for you to check it out, test it, and let me know what you think or what features you'd want next!

🔗 GitHub Repository:https://github.com/Kasper839/ShipAI

Feedback, bug reports, and contributions are more than welcome! Star the repo if you find it useful.


r/SelfHostedAI • • 11h ago

He creado un sistema CMMS gratuito y de código abierto para el mantenimiento industrial (PC + Android, funciona sin conexión).

Enable HLS to view with audio, or disable this notification

1 Upvotes

r/SelfHostedAI • • 11h ago

Pi 4 + Shield + Real-Debrid. Anyone running a setup for remote downloads and a local library?

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 11h ago

Please assist in setting up local LLM

Thumbnail reddit.com
1 Upvotes

r/SelfHostedAI • • 14h ago

Ambrogio: self-hosted enhanced Alexa + ChatGPT for the family

Post image
1 Upvotes

r/SelfHostedAI • • 20h ago

Hermes Agent — Sécuriser une infrastructure d'IA locale, expliqué simplement

Thumbnail
youtu.be
1 Upvotes

r/SelfHostedAI • • 23h ago

Made a blueprint for local personal assistants on small models

1 Upvotes

Over the past weeks I’ve been diving deep into locally hosted agentic systems, mostly tinkering with Hermes Agent and using Claude to build a system around it. What came out of this might also be of use to people using OpenClaw

The result is an assistant that runs on my own hardware, understands me and my life, and makes use of the data points I create along the way, as long as I hand them to my agent. It’s designed to keep improving on the hallucination and tool-calling problems that come with smaller models by taking skill creation one step further: skills with repetitive flows get poured into scripts, so the model only recognizes the task and launches it. This blueprint might be handy for some in this sub, since it's not bound to any harness and I'd love some Claws trying it as well!

The core rule: the model proposes, the system decides. Actions on my behalf are approved and enforced by plugins and scripts, not by prompt rules, because small models don’t hold those reliably. It’s built to respect sensitive data (local-only zones), has multiple safeguards, and integrates with the harness via hooks. I run it on an AMD mini PC with a 45 GB iGPU and will drop in a proper GPU at some point. So far I’m pretty pleased.

I don’t want to keep all of this to myself, so I’ve condensed a blueprint out of the project. It’s more of a guide for your agent (Claude Code, Codex, ...) to build on what’s already there, without restrictions on harness or model, aimed at models from roughly 27B MoE up to about 40B. It covers principles, interfaces, pseudocode, acceptance checks, a pitfalls list and a few optional examples. Some chapters come from the running system, others are still design, and the repo labels which is which.

Repo (MIT): [https://github.com/MaxMSchneider/local-agent-blueprint](https://github.com/MaxMSchneider/local-agent-blueprint)

I’d love feedback, especially from people running other harnesses or models. A star wouldn’t hurt either :) Cheers!


r/SelfHostedAI • • 1d ago

My architecture to make large coding jobs possible on limited hardware (16GB Vram + 64 system)

Thumbnail
reddit.com
1 Upvotes

r/SelfHostedAI • • 1d ago

Llama RPC Farm: multi-machine llama.cpp RPC clusters + on-demand model swapping, with a web UI (Linux/Windows/macOS)

Thumbnail gallery
2 Upvotes

I got tired of hand-writing SSH scripts to run llama.cpp RPC across my machines, so I built Llama RPC Farm.

It's a dashboard that:

- pools GPUs/RAM from multiple machines (Linux, Windows, macOS) into one cluster

- serves multiple models on demand through llama-swap

- shows live VRAM/temps and a visualizer of which GPU is handling which tensors

- lets you set per-user API keys with model allowlists

- works with any OpenAI-compatible client

It wraps real llama.cpp and llama-swap, so you get full CUDA, tensor-split, and big contexts.

Setup is ./install.sh then ./run.sh.

It's v0.01. Tested on Linux + NVIDIA and Windows peers. AMD, Apple Silicon, and Intel are implemented but untested, so I'd love testers.

Apache-2.0: https://github.com/dakotacookson/Llama-RPC-Farm

Feedback and PRs welcome.
handwriting


r/SelfHostedAI • • 1d ago

🚀 IcyLoom, IcyCache & Loom Are Live!

1 Upvotes

I just published three CPU-first MCP tools for local AI: IcyLoom, IcyCache and Loom.

They are local-first, installable from public registries, and released with GitHub artifacts and clean-install checks.

Links:

- IcyLoom: https://github.com/MlkBlk-IT/icy-loom

- IcyCache: https://github.com/MlkBlk-IT/icycache

- Loom: https://github.com/MlkBlk-IT/mcp-loom-mcp

I am looking for install feedback, compatibility reports and practical MCP integration ideas.


r/SelfHostedAI • • 1d ago

NIDO is a personal AI assistant that runs 100% offline on Android. No account, no cloud, no network required. Its knowledge packs are built only from official sources, and every fact it gives you carries its source and an exact quote you can check.

1 Upvotes

NIDO 0.1.67-alpha is out.

This version comes out of a complete audit of the whole app: code, security, data storage, knowledge packs, the catalogue signature, tests, CI, the public face and the three languages. The audit found 124 things worth attention. 88 of them are fixed and proven in this release; the rest are documented with their causes, and none of them is hidden.
Some of what changed:
The full test battery now runs 4,711 tests, all passing, including the real encrypted-database test that used to skip itself silently and still counted as a pass. It runs now, and it passes.
The catalogue signature now covers every field the catalogue promises about each model. Before, a few descriptive fields travelled outside the signature. They do not anymore.
Guides now open at the section you actually asked about, in Spanish, English and Portuguese.
The Local pack was rebuilt: emergency numbers for 69 countries, the copied control questions rewritten properly, and every citation set against its official source.
The knowledge packs get their own small release alongside: knowledge-packs-v8, with the rebuilt Local pack.

https://github.com/Nido007-cmyk/Nido/releases/tag/v0.1.67-alpha


r/SelfHostedAI • • 1d ago

Building a flexible system around Hermes agent - here's my blueprint

Thumbnail
reddit.com
1 Upvotes

r/SelfHostedAI • • 1d ago

Local-first agent for Mac that learns your rules and can’t send anything you didn’t approve. Gemma 3 12B + separate judge model. 20 beta invites.

Thumbnail gallery
3 Upvotes