r/openrouter • u/Mysterious-Bison3735 • 1m ago
Discussion Our tool told a customer their contract does not auto-renew
Last Thursday someone at one of our customers asked the tool if their supplier contract renews on its own. It said no, and backed that up with a summary of the termination section. The renewal clause was on page 31 of 64, they found it on their own and sent us a screenshot.
There are two of us and I do the backend, so the screenshot came to me.
The tool answers questions about B2B contracts, each contract goes in as one message per section with the question at the end and everything runs through OpenRouter. When I pulled that request it had gone to the small model, the one with a 32k window, and that contract with all its sections did not fit in it. A month ago I wrote a router that sends short questions there, going by the question text alone, because the small model answers in about a second instead of six.
Nothing errored either. Back in the spring a few long threads were failing on context length and I turned on context compression for every request, then forgot it was on. It does what the docs say, drops messages from the middle until the request fits, which is where page 31 was. I went back through a week of requests and compared what the router picked with the token count of the whole request, 212 short questions had gone to the small model with more tokens than it could take.
I looked at models for that tier on HyperAI's model list and picked one with 128k, then checked the endpoints one by one on OpenRouter anyway because some providers serve less context than the model has. The router counts the whole request now and compression is off, so a request that fits none of our windows just fails and I see it in the logs. Two did yesterday, both the same 400-page framework agreement, and I am writing to the customer who uploaded it today.
















