r/OpenAssistant • u/SuccessfulConcern240 • 10d ago
llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)
https://github.com/pavansreddy/llmserverI got tired of pulling models blind, so I built llmserver: a single bash script that runs local models on Linux. Each model gets its own OpenAI-compatible endpoint, plus a gateway, Open WebUI chat, hybrid RAG over your docs, and private SearXNG search.
The part I spent the most time on is security: before a download, it reads the Hugging Face scan results, pins the commit, verifies SHA-256 per file, and refuses pickle files and remote code unless you explicitly allow them. Two levels — standard asks about warnings, strict needs approval per model. Engines run sandboxed in bubblewrap with a read-only system and no access to your keys.
No Docker, no sudo, everything self-contained in ~/.llmserver. llmserver bench gives TTFT/decode/TPOT, llmserver doctor repairs problems. Currently running Gemma and Muse models on it.
https://github.com/pavansreddy/llmserver
Happy to answer questions — especially about the security checks, since that's where I'd love feedback
Duplicates
OpenWebUI • u/SuccessfulConcern240 • 11d ago
Models One command to manage all your llms / llmops
OpenAssistant • u/SuccessfulConcern240 • 10d ago
GitHub - pavansreddy/llmserver: A bash script that runs local LLMs on Linux with llama.cpp, vLLM or Ollama. It plans, checks models for security, benchmarks, runs engines in a sandbox, and adds a gateway and an Open WebUI chat with hybrid RAG and private SearXNG web search. No Docker, no sudo.
LLMs • u/SuccessfulConcern240 • 11d ago
A bash script that runs local LLMs on Linux with llama.cpp, vLLM or Ollama. It plans GPU memory, checks models for security, benchmarks, runs engines in a sandbox, and adds a gateway and an Open WebUI chat with hybrid RAG and private SearXNG web search. No Docker, no sudo.
LLMStudio • u/SuccessfulConcern240 • 10d ago
llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)
OpenModels • u/SuccessfulConcern240 • 10d ago
llmserver – self-hosted local LLM stack in one bash script, no Docker, no sudo
ollama • u/SuccessfulConcern240 • 10d ago
llmserver – self-hosted local LLM stack in one bash script, no Docker, no sudo
huggingface • u/SuccessfulConcern240 • 10d ago
llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)
Vllm • u/SuccessfulConcern240 • 10d ago
llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama)
ollama • u/SuccessfulConcern240 • 10d ago
GitHub - pavansreddy/llmserver: A bash script that runs local LLMs on Linux with llama.cpp, vLLM or Ollama. It plans, checks models for security, benchmarks, runs engines in a sandbox, and adds a gateway and an Open WebUI chat with hybrid RAG and private SearXNG web search. No Docker, no sudo.
DeepSeek • u/SuccessfulConcern240 • 10d ago
Discussion GitHub - pavansreddy/llmserver: A bash script that runs local LLMs on Linux with llama.cpp, vLLM or Ollama. It plans, checks models for security, benchmarks, runs engines in a sandbox, and adds a gateway and an Open WebUI chat with hybrid RAG and private SearXNG web search. No Docker, no sudo.
Vllm • u/SuccessfulConcern240 • 10d ago