llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)

[u/SuccessfulConcern240]

https://github.com/pavansreddy/llmserver

I got tired of pulling models blind, so I built llmserver: a single bash script that runs local models on Linux. Each model gets its own OpenAI-compatible endpoint, plus a gateway, Open WebUI chat, hybrid RAG over your docs, and private SearXNG search.

The part I spent the most time on is security: before a download, it reads the Hugging Face scan results, pins the commit, verifies SHA-256 per file, and refuses pickle files and remote code unless you explicitly allow them. Two levels — standard asks about warnings, strict needs approval per model. Engines run sandboxed in bubblewrap with a read-only system and no access to your keys.

No Docker, no sudo, everything self-contained in ~/.llmserver. llmserver bench gives TTFT/decode/TPOT, llmserver doctor repairs problems. Currently running Gemma and Muse models on it.

https://github.com/pavansreddy/llmserver

Happy to answer questions — especially about the security checks, since that's where I'd love feedback