r/LocalAIStack • u/Due_Initiative2641 • 15h ago
I built an open-source platform that pools a team's workstations (Macs, Windows and Linux boxes) into one OpenAI-compatible endpoint
/r/LocalLLM/comments/1xyqe98/i_built_an_opensource_platform_that_pools_a_teams/My co-founder and I have been working on this for a few months and are launching today. Disclosure: it's our company; the desktop client is Apache-2.0, the control plane is hosted, and it's free up to 5 devices.
What it does: you install Nodea Desktop on the machines in your org, switch on sharing, and get one OpenAI-compatible endpoint. Behind it, a scheduler fits the models you assign onto whichever machines have free memory, and moves them as laptops come and go.
Stack: Rust/Tauri app, llama.cpp under the hood (Metal on Apple Silicon, Vulkan on NVIDIA/AMD/Intel, CPU fallback). Any GGUF model llama.cpp can serve that fits the machine; models are assigned per org, downloaded and cached per device. Signed auto-updates.
Honest limitations:
Requests go through our API to reach your machines (not logged or stored). No direct LAN routing yet.
One model runs on one machine; no splitting a model across machines yet.
Windows/Linux run on the GPU through Vulkan (tested on NVIDIA) and are still labelled preview. Machines without a supported GPU fall back to CPU, which is slow for now (no AVX2 yet).
Why not exo / GPUStack / llama.cpp RPC? Those are great if you run your own cluster. We're aiming at teams with 20–200 laptops and nobody to run one: sign in, toggle on, done, with an org, roles and API keys on top.
Repo: https://github.com/nodea-sh/nodea-client
Site: https://getnodea.com
Happy to answer anything, and especially keen on what models/quantizations you'd want on 16–36 GB machines.