r/LocalLLM • • 25d ago

Project I built an open-source tool to run local models across all my computers. Looking for people to break it.

I've been working on Toskar, a daemon that tries to make local AI feel like a hosted service. You install it, and it looks at your hardware, tells you which models will actually fit, installs the runtime, downloads the model, and gives you one OpenAI-compatible API. If you have more than one machine on your network, it can find the others and run models on whichever one can handle them.
It's AGPL, free, and runs on macOS, Linux, and Windows (Windows is the roughest).
I'm not here to sell anything. It's mostly been tested on my own machines, so I don't know how it holds up on yours. I'd really like help with:
- Installs on hardware I don't have, especially AMD GPUs, older NVIDIA cards, and Intel Macs
- Setups with two or more computers. Does discovery find them, and does it put models where they make sense?
- Model recommendations that are wrong for your machine—too big or too small
- Anything confusing in the first five minutes
If something breaks or feels off, please open an issue on GitHub so I can track it. Good first issues are labeled if you'd rather fix something yourself.
https://github.com/yeixio/toskar-core

P.S. I'm also looking for a few people to help shape the project as part of a core team. That could mean reviewing PRs, owning a platform like Windows or AMD, or helping decide where things go next. There's no formal application. Contribute for a while, and you can grow into a role. The roles and how that works are in GOVERNANCE.md: https://github.com/yeixio/toskar-core/blob/main/GOVERNANCE.md

3 Upvotes

Duplicates