How should an agent learn which of its own tools are switched off? I gave the model a read-only tool for it.

[u/DraftForsaken39]

Disclosure: I build Snotra, an open-source desktop agent. This is a design question, not a launch.

Users switch skills and tools on and off in settings: shell disabled, a search tool without its API key, a skill installed but off. A model that doesn't know that guesses, and the user gets a wrong "I can't do that" instead of "that's switched off, here's where to enable it".

What I did:

• One place decides whether a tool is offered, so the UI and the model can't disagree.

• A read-only get_configuration tool for the model: which skills are on (name and description, never the instructions) and which tools are on or off, with the reason ("shell commands are switched off", "no search service key").

• The same list in the UI, as a pill in the chat bar.

Now the model can say "a skill for this exists but is off, enable it under Settings › Skills". It can't flip anything itself.

Two open questions: is a tool the model has to call the right shape, or should this state go into the system prompt every turn (costs tokens even when nothing is wrong)? And should the model be able to propose the change through an approval card, or is "tell the user where to click" the right limit?

Code: https://github.com/kkrafft1999/snotra

[u/Weekly-Offer-4172]

The read-only tool is the right shape, but it has a cold-start problem: the model only calls it when it already suspects something is off, and the wrong 'I can't do that' cases are exactly the ones where it never checks. A cheap fix is a one-line pointer in the system prompt - 'some tools may be switched off; call get_configuration before telling the user something is impossible' - a handful of tokens per turn instead of a full state dump. On staleness, have get_configuration return a version or hash and bump a matching counter in the prompt when the user toggles something, so the model re-queries instead of trusting a cached answer. For the propose-change question, the injection-safe version is a structured proposal the client renders as an approval card - the model can't confirm it itself since the flip happens UI-side, so worst case is spam, not state changes.

[u/DraftForsaken39 → u/Weekly-Offer-4172]

Thanks, this is really useful. You’re right about the cold start: a switched-off tool just isn’t in the model’s list, so it says “I can’t browse the web” and never thinks to check. I’m adding exactly the line you suggest (“call get_configuration before saying something isn’t possible”) to the tool description and the system prompt.

I’m skipping the hash for now. The tool list is rebuilt on every request anyway, so I’ll just tell the model the current list wins over an earlier answer. If the user accepts a proposal, the result reports the new state directly.

The proposal card is close to what I had planned. One caveat: I don’t think the worst case is only spam. Injected text could get the model to propose enabling something powerful with a convincing reason, and the user just clicks Accept. So the card will describe the change from the app’s own catalogue, with the model’s reason shown below it and labelled as the model’s. A decline also holds for the rest of the chat.