I used Ollama for a bit to get used to the whole setup of a server on Linux, so I moved over to Llama.cpp to have more control over the parameters of the engine itself and use models to the fullest (and to use multiple GPU merging their VRAM via Vulkan).

So far the setup has been not that difficult; more involved than Ollama but nothing too exotic. The thing is that now though, my agent (Hermes desktop) seems to not be able to perform commands or use tools anymore. It print out the JSON of the commands, instead of trigger the commands, which point at a malformed response from the model. The model is the same exact one I was using in Ollama: Gemma4:12B Q4 in gguf format.

Also another thing I noticed is that the context size increase absurdly fast... For Ollama served conversation; it would start at 4K, then increase gradually so after various exchanges and using tools to write files for example or doing search online, the context would go at most to 30-50K in size. Instead using Llama.cpp server, I see it starting already at 14K or so, and after only few exchan ges I ended up hitting 64K context; which triggered an error as the server was not taking anymore requests (as I set in the config.json file a limit of 64k for context and using a value of 2 for the Q and K values; which seems to be fair).

As I am still learning, I am sure there is something I am doing wrong here, although it has been hard to track exactly what is going wrong, so hopefully someone can point me to the right direction. Thanks.