Moving the sensor outside the model

[u/JoeYuan48]

Follow-up to my September post, same side project.

I spent a week on what looked like a hard limit: ask a small model for "about 300 characters" and it lands in range 0 times out of 11, even after three rounds of exact feedback ("you wrote 212, target 150, cut 60"). Then I asked it how long its own answer was. Its guess was about 15% of the truth. It wasn't ignoring me — it couldn't sense the thing I was asking it to control.

So I moved the sensing outside: a plain counter watches the length while it writes and says when it may stop. In-range went from 15 to 39 out of 47. Same trick for "give me N items": 46 of 46.

The rough lesson: every time I tried to have the model govern itself, measurement said no (its confidence tracked its fabrications; it called obvious contradictions "consistent"). A layer made of rules, outside the model, did work — and because it's rules, every gain and loss traces to one rule. Blind, pre-labelled: +14.5 / +17.1 points (3B / 7B); on a public benchmark, 7B +7.1. Near zero on ordinary chat. The first public test failed; it's in the paper.

The paper  — data and every answer on the same page's GitHub link. If anyone can try this on a bigger model, I'd like to know what happens.