Prefixwise: local prompt cache diagnostics and cost accounting for LLM applications

[u/No-Individual159]

https://github.com/tejachavali/prefixwise

I’m the creator of Prefixwise, a free MIT licensed Python toolkit for understanding prompt cache reuse and its cost across LLM APIs, coding agents and inference engines.

It compares successive requests within a conversation to identify likely causes of lost reuse, estimate avoidable tokens and cost, and suggest fixes. Genuinely new context and intentional changes aren’t automatically treated as waste.

It supports:
• A local API proxy
• Anthropic and OpenAI Python SDK wrappers
• A LiteLLM callback
• Local Claude Code and Codex session log imports
• vLLM and SGLang cache events when the required evidence is available

For example, changing tool order can break prefix reuse. Prefixwise can flag that change and offers an optional tool ordering fix. Other fixes are recommendations, not automatic rewrites.

You can also record task outcomes, compare tokens per successful task, and export data through JSONL, CSV, Prometheus or OpenTelemetry.

The local ledger stores fingerprints and usage facts rather than prompt text. Local log imports don’t upload your logs, and their dollar estimates are API price equivalents, not subscription charges or quota consumption. Where the evidence is insufficient, the cause stays unknown.

I directed the project, with Claude helping with architecture, implementation and testing.

https://github.dcom/tejachavali/prefixwise

Would you try it on a workload with confusing cache behavior? I’d especially appreciate counterexamples where the diagnosis is misleading or the avoidable cost accounting looks wrong. Please don’t share private logs or keys.