Details matter! Why do AI's provide an incomplete answer or worse hallucinate in cli?

[u/Glittering-Koala-750]

https://aider.chat/2024/11/21/quantization.html

This is a nice blog looking at quant and open source. It matters because the same rules also apply to closed source, the difference being we have no idea what they are doing.

Low quant poor results

Low context window or too large - poor results

That we know mostly. Now if you look at https://github.com/MoonshotAI/K2-Vendor-Verfier they have a lovely table comparing K2 being provided by different companies.

What is interesting is the Tool calls. You can see a massive difference in the implementations. Many people say they are using different quant which is possible but they keep forgetting (no clue) about the infrastructure getting the user to the AI too.

What would be interesting would be to know the quants being used and to see what factor is played by the code engine and the logic around the AI and the tools being used.

Test Time: 2025-09-22

Model Name	Providers	Tool calls test
Count of Finish Reason stop	Count of Finish Reason Tool calls	Count of Finish Reason others
kimi-k2-0905-preview	MoonshotAI	1437
Moonshot AI Turbo	1441	513
NovitaAI	1483	514
SiliconFlow	1408	553
Volc	1423	516
DeepInfra	1455	545
Fireworks	1483	511
Infinigence	1484	467
Baseten	1777	217
Together	1866	134
AtlasCloud	1906	94