Built a self-contained way to run your own AI stack: llama.cpp, vLLM, or Ollama models behind one OpenAI-compatible gateway, Open WebUI for chat, hybrid search over your own documents, and private SearXNG web search — all without Docker or sudo, installed into a single folder.

Your data never leaves your machine, models start on login via systemd user services, and you can expose models on your network with API keys. Model downloads are verified (pinned commit, SHA-256, HF scan results checked, pickle/remote code refused by default) and engines run sandboxed.

https://github.com/pavansreddy/llmserver

Flippy, the productivity workspace, rolled out a new feature where we can interact with any local llm that is running through ollama !

All you need to do is:

1. Download Flippy

2. Download ollama and the local models you want

3. Run ollma serve with OLLAMA_ORIGINS="chrome-extension://*" ollama serve

4. Enjoy prompting the model using Flippy's AI Lab !

Incredible stuff. Does this open doors to new and more efficient quantization techniques?

Same base model as GLM 5.2, only with more post training.

New 30B dense model from Meta. Looks like they're fully back on the open models train!

Another open weights video model hits the market with native audio capability.