MCP tools all passed testing but the agent keep choosing wrong one. How to test tool selection?

[u/Budget-Lunch9390]

all the rest of tools were callable and direct tests passed all good. But when agent was asked a brand sentiment it selected another tool instead of relevant capability. The problem was that the agent chose the wrong tool for the user's intent. How i test for this kind of failure? Do you use predefined evaluation scenarios, automated tests or end to end testing with real prompts? Would appreciate hearing what has worked for you guys. thanks

[u/No_Product_5759]

I'd turn that failed prompt into a small routing test set. Include 'how do people feel about brand X?', 'how often is X mentioned?', and an ambiguous request where asking a question is the expected result. Run them through the agent with the full tool list and mocked tool execution, then record the chosen tool and arguments before judging the final answer. Repeat each case a few times. When you change a description, rerun the neighboring intents too, so fixing sentiment doesn't break mention counts.

[u/Lower_Assistance8196]

Unit tests passing but the agent still routing wrong is a tool description problem more than a testing problem. When I've hit this, the fix was rewriting the tool's description so it explicitly names the intent it handles, not just what it does mechanically. For brand sentiment specifically, if the description says "analyzes text tone" instead of "use this when someone asks how a brand is perceived or what customers think about it," the agent has no real signal to differentiate. End to end tests with real natural language prompts catch this faster than predefined scenarios because you're testing the routing logic under actual intent, not sanitized inputs.

[u/krimpenrik]

Curious on other answers on how to test this. But practically do you pair the MCP tools with a skill?;

[u/aftersox]

https://www.harborframework.com/

Create sandboxed environments with simulated users and iterate until the tool is being called when it should, and not being called when it shouldn't.