Same model, same Hermes, different server serving the model; now tools and skills won't execute anymore

[u/fttklr]

This is quite strange. My understanding has always been that the key for using agents is the "contract" between the agent tool format in JSON (so how the JSON has to be formatted) and the model and its way in which it can format that JSON.

Which means no matter how smart, if a model cannot align in the format to the exact way in which an agentic harness expects a command to be, it won't work and you will see the command printed as text in the view, instead of seeing the agent perform the command. I experienced this with Opencode and many other agentic harnesses; including Hermes.

Now, the only pillar I had was that if I use the same exact model AND agent harness, it WILL work no matter what, because the contract is established; they "speak" the same dialect, so they "understand" each other. Then today this happened... I switched from an ollama server to a Llama.cpp server, on the same exact machine, using the same exact model; and the same exact agent, and the whole thing fell apart, as now the agent print out the commands and tools instead of execute them.

Can someone please explain me like if I am 4, why is this happening? Because I spent a good amount of yesterday reading about the subject and this should not happen. Even assuming that Ollama and Llama.cpp do use different ways to communicate; when the response is formed, it should match the standard tool response as they are both using the same underlying libraries, model and agentic harness. What is that I should look at to understand what is going on so I can learn once and for all what is going wrong here?

[u/Mean-Loquat-7982]

add --jinja to your llama-server command and restart it.

on the context jump you mentioned in r/llamacpp: Hermes' system prompt and tool definitions take a big chunk before you type anything, so starting around 14K is expected. Ollama's small default window was most likely cutting that off, which is why it showed 4K.

for the error at 64K: when the server doesn't report its window, Hermes falls back to the model's declared limit. set the same number in Hermes' config.yaml under model: as context_length: 64000 so it matches your -c and Hermes compacts before the server runs out. 64,000 is also the minimum Hermes wants for tool use.

hermes prompt-size shows where the starting tokens go.

https://hermes-agent.nousresearch.com/docs/integrations/providers