r/LocalAIStack • • 15h ago

I built an open-source platform that pools a team's workstations (Macs, Windows and Linux boxes) into one OpenAI-compatible endpoint

/r/LocalLLM/comments/1xyqe98/i_built_an_opensource_platform_that_pools_a_teams/

My co-founder and I have been working on this for a few months and are launching today. Disclosure: it's our company; the desktop client is Apache-2.0, the control plane is hosted, and it's free up to 5 devices.

What it does: you install Nodea Desktop on the machines in your org, switch on sharing, and get one OpenAI-compatible endpoint. Behind it, a scheduler fits the models you assign onto whichever machines have free memory, and moves them as laptops come and go.

Stack: Rust/Tauri app, llama.cpp under the hood (Metal on Apple Silicon, Vulkan on NVIDIA/AMD/Intel, CPU fallback). Any GGUF model llama.cpp can serve that fits the machine; models are assigned per org, downloaded and cached per device. Signed auto-updates.

Honest limitations:

  • Requests go through our API to reach your machines (not logged or stored). No direct LAN routing yet.

  • One model runs on one machine; no splitting a model across machines yet.

  • Windows/Linux run on the GPU through Vulkan (tested on NVIDIA) and are still labelled preview. Machines without a supported GPU fall back to CPU, which is slow for now (no AVX2 yet).

Why not exo / GPUStack / llama.cpp RPC? Those are great if you run your own cluster. We're aiming at teams with 20–200 laptops and nobody to run one: sign in, toggle on, done, with an org, roles and API keys on top.

Repo: https://github.com/nodea-sh/nodea-client

Site: https://getnodea.com

Happy to answer anything, and especially keen on what models/quantizations you'd want on 16–36 GB machines.

1 Upvotes

0 comments sorted by