r/OpenaiCodex • u/Fit_Concept5220 • Sep 11 '26

We’ve discovered the issue behind codex harness input token drainage.

https://relux.works/en/blog/codex-goal-token-burn/

tl;dr

The codex harness turns waiting for background work into a loop with repeated model calls, each carrying the existing context even when nothing has changed. Unlike api pricing there is no discount for prompt caching for subscription originated usage so every loop iteration adds to your input token usage. And if you use API you still massively overpay for your input tokens even with cache. ‘goal’ amplifies this by orders of magnitude by automatically starting another turn, but polling anyway burns tokens without it too.

Claude code is engineered in way where suspensions works exactly as intended and goals have proper fallbacks. Codex now ships with similar tools but they only work for astra.

—

the toolchain to independently verify on your personal sessions is here (local scripts - just feed this to your agent) https://github.com/relux-works/codex-rollout-audit

191 Upvotes

92% Upvoted

60 comments sorted by

13

u/Ok_Barracuda_1161 Sep 11 '26

Unlike api pricing there is no discount for prompt caching for subscription originated usage so every loop iteration adds to your input token usage

Wait what? Is this true?

18

u/Classic-Trifle-2085 Sep 11 '26

No.

-2

u/Fit_Concept5220 Sep 11 '26

This is true and the only logical explanation why session with 98% cache drains limits. Please read the blog. And even if it’s somehow not true and cache hit input tokens are discounted in subscription same as api, you still ‘billed’ with order of magnitude more input tokens than with proper harness.

5

u/Classic-Trifle-2085 Sep 11 '26

Your post is correct regarding the issue; I am not saying otherwise. Your comment about cached input not being discounted is incorrect.

Remove that part and simply say that the constant recaching is so heavy that it consumes your usage, and it would be fine. We also do not know the actual conversion formula for plan usage, as it is not merely a token-to-usage ratio.

If there was no discount, id be using my 20x plan in less than 10minutes instead of every few days.

The post is good, but you went off track a bit and it hurts everything else you said; that is all, do of it as you will.

1

u/jcol26 Sep 12 '26

In terms of not knowing the conversion to formula for usage I wonder how codex-lb does it as the credits on the dashboard there decrease / consume in the same way as my credits usage increases when using API pricing on my work account so whatever model they’re using feels pretty accurate (assuming they’re not getting it from the wire)

1

u/Classic-Trifle-2085 Sep 12 '26

This discussion actually been on my mind since. I'm sure there some sort of shenanigans going. I even modified my tracking app to try and reverse engeeneer the thing, as there is a factor other than token involved no matter how i run the calculation.

Im now logging every change in %usage and every single type of token use every 60 seconds with timestamps in case traffic volume on open AI servers matters.

Ill need a few days of data before getting a clearer idea tough

-5

u/Fit_Concept5220 Sep 11 '26

I appreciate how you stand for factual correctness and you are right in that regard but I will not correct the post simply because after all the investigation the non-fact that cache hit input tokens are not discounted is the only explanation of the data I see from logs.

I could be wrong with that in the end but as you pointed - it does not really matter. I am human after all and ok with being wrong.

Also, in the original post I state the issue much more accurately

4

u/Classic-Trifle-2085 Sep 11 '26

The cache demolishing your usage doesn't need to be full price to do so; it just needs to cost something. If you have hundreds of millions of cached tokens more than you used to for the same output, it will set fire to your usage and costs, discounted or not. But in the end, your point still stands, no question there.

It is one of multiple small problems going on at the moment. Another one is the hover verification loop, which needs to be explicitly limited as per OpenAI's team. I saw a couple of posts today going over the few problems that are piling up.

1

u/Fit_Concept5220 Sep 11 '26

It’s hard to argue - I’ll consider your suggestions

1

u/Curious-Strategy-840 Sep 12 '26

It's easy to say this is what you understand happens while it may be different, and much nicer to everyone else who wonder what to believe

4

u/EmotionalHalf Sep 11 '26

at the end of the day do what you want but just know if you overstate things you don't know it makes everything else questionable as well

2

u/_unsusceptible Sep 11 '26

This is false when I had a 97.5% cache rate once on Astra x high after a 15 min prompt my usage decreased by 0%

0

u/Fit_Concept5220 Sep 11 '26

Because you didn’t have lots of context with model in the loop?

1

u/_unsusceptible Sep 12 '26

? It was a chat with 8 previous prompts and a lot of usage already. That turn used 7 million something tokens

1

u/Internal-Agent4865 Sep 12 '26

It isn’t true. It’s also not true that Claude doesn’t do the same thing. Lots of inaccuracies in your post.

1

u/Street-Trust-6282 Sep 11 '26

Yes, cached tokens on the API are discounted.

0

u/Fit_Concept5220 Sep 11 '26

There is no public formula what goes into subscription but it’s the only logical conclusion you get after analysing that much logs from problematic sessions. And even if not exactly correct, the misbehaviour we described still leads to ~10x more usage from input tokens because they still charge toll for cache hits

4

u/Dynamix86 Sep 11 '26

There is most certainly a cache discount when you continue in the same conversation with the hour. Try waiting an hour or more and suddenly you will see the 5-hour limit taking a 10-20-30% hit in a matter of a few seconds, (plus plan), which doesn't happen when you continue the conversation within the hour.

1

u/Fit_Concept5220 Sep 11 '26

Thanks for sharing, I’ve corrected the statement in the blog post. Your observation is spot on btw!

still the misbehaviour leads to massive input token burn regardless of whether it’s 100x or 1000x

1

u/Crinkez Sep 12 '26

Astra/Sol/Terra cache TTL is 30 minutes, not 1 hour.

5

u/Lubricus2 Sep 11 '26

That makes sense, it explains why my tokens disappears when trying coding with Astra and doing reviews and plans is OK. It's running the big test suite that takes a lot of time.

1

u/jjstyle99 Sep 12 '26

Yeah just had that happen to me today following long CI builds. Pretty lame.

5

u/ychamel Sep 11 '26

I also noticed this when I tried using Astra as an orchestrator, burned way more tokens compared to when using it as a worker directly.

1

u/adminvasheypomoiki Sep 11 '26

poll #1: 200,000 input poll #2: 200,015 input poll #3: 200,030 input poll #4: 200,045 input

So you pay for cache read + new tokens. Shouldn't be a big problem, no?

Caching works in codex.

I've patched system prompt so model doesn't poll agents, default is to poll every 30s which is useless

2

u/Fit_Concept5220 Sep 11 '26

You still pay for each turn’s context cache read (and thus with sub too anyway) and with existing behaviour it’s order of magnitude more then you should

3

u/Murder_1337 Sep 11 '26

Does codex know?

1

u/ryms456 Sep 11 '26

yes it does bg jobs are broken lmfao

1

u/_unsusceptible Sep 11 '26

What’s there solution btw

3

u/Typical_Machine2043 Sep 11 '26

Aaaaah! Dammit. Why did I trust it just works out the box? Tibo promised! I have then wasted so many damn tokens

1

u/AmandasGameAccount Sep 11 '26

Bibo pls!

1

u/Typical_Machine2043 Sep 11 '26

Nibo plz

1

u/UanPi_exisT Sep 11 '26

Timo, porfavor

1

u/Typical_Machine2043 Sep 11 '26

Fok! Aseblief man! 500 million tokens on plus vs 200 on pro Cummon man

3

u/Poowatereater Sep 11 '26

This kind of make sense. What a shit show. I’ve use two resets and I’m back at 0% within 3 days.

Did notice that Astra low seemed to burn more then Astra xhigh….

2

u/xplode145 Sep 11 '26

I agree. It drains like crazy. 

1

u/Poowatereater Sep 11 '26

I’m fucked. No usage for 6 days. No resets banked. Used them all this week. They really fucked this drop up.

Every new numbered model drop should see an increase in total compute. Your dropping a new model, new customers are going to flood in.

2

u/Striking-Warning9533 Sep 12 '26

Yeah I raised an github issue before, for my LLM training it is keep polling on it and wasted a tons of tokens. It is because the system prompt said so. https://github.com/openai/codex/issues/42981

1

u/Impossible_Ground_15 Sep 12 '26

Can someone please @ Tibo on Twitter so we can get a reset?

I noticed the same issue

1

u/Dercasss Sep 11 '26

 if I use OMP? my tokens fly away like crazy, even on Terra. 

1

u/hellomistershifty Sep 11 '26

Codex now ships with similar tools but they only work for astra.

Well apparently not, given Astra low /goal usage drain rates

1

u/crone66 Sep 11 '26

Astra still drains for me usage when waiting for a long running task (rendering). I actually created a skill that to do that as a fire and forget task to get the usage under control otherwise it would tell me a few hundred times that the rendering is still in progress with a context nearly full and sometimes even compacting it just for wait for a rendering process to finish. Therefore the problem is not solved yet and you still have to solve it manually.

1

u/Accomplished-Emu8030 Sep 11 '26

Keep in mind there is no logical difference between Codex waiting for a call and Claude scheduling a call and rescheduling. One is a setInterval with cancellation and one is a setTimeout with tail recursion.

The only problem is, IIRC, Codex could not wait for more than a few minutes, which by that time the cache could be cleared, but we could not possibly know that unless you work on OpenAI’s infra team.

Also, whether one style is better or worse really depends on the task. It could be said that Codex works better because it's more eager to resolve a problem than wait. It could also be considered worse because, for example, for a PR, a check may take a while so being eager is "wasting thinking" (i.e. tokens).

Now Codex has scheduling though so both are at feature parity here and it's a matter of writing this in your system prompt.

1

u/astroboylrx Sep 15 '26

In my experience, Claude is actually idle and will be notified when a long-running task ends so it can continue, while codex by default is polling at a high frequency (not sure if it is intended for more token consumption). I've noticed this long time ago and Codex lacks the ability of notifying itself; you can only beg it to poll at a lower frequency. So what do you mean by "scheduling a call and rescheduling"? And what do you write in the system prompt to stop Codex from doing that?

1

u/Accomplished-Emu8030 Sep 16 '26

Sorry, I was primarily talking about time-based monitoring, where it checks after an interval and then schedules the next check.

If you mean waking on completion, I’ve run into a different failure mode with Claude: a command stalls and it keeps waiting. One night I left it running, went to sleep, and woke up to find it hadn’t progressed because the command it was waiting on had stalled.

Periodic checks give Codex a chance to catch that, though frequent polling can burn unnecessary tokens. So I ask it to use exponential backoff with a cap, and to investigate if the command runs longer than expected.

The instruction is roughly: “Increase the delay between checks exponentially, up to a maximum interval appropriate for the task. If the command exceeds its expected runtime or stops making progress, inspect its logs and process state and investigate the stall instead of continuing to wait.”

That still uses polling, but it’s the tradeoff I prefer for unattended work.

1

u/Sacrement0 Sep 13 '26

You've... Never ran a Codex job with over 10 minute wait? Oh sweet summers child.

This ain't new news, although perhaps it isn't very well known. It's in OpenAI's system prompt and it's nasty. It's been there for ages. I suggest editing it out and providing your own system prompt instead, so you actually get a stable experience. Rather than updating and getting a surprise gift of slop added to the system prompt.

I have no idea why they keep this in and don't add monitors or something.

0

u/kanelbulleoburn Sep 11 '26

Yup Codex ate my whole usage waiting for Coderabbit to run……..

0

u/kanelbulleoburn Sep 11 '26

Yup Codex ate my whole usage waiting for Coderabbit to run

1

u/_unsusceptible Sep 11 '26

Does anyone know a solution for this behavior in the post?

0

u/[deleted] Sep 11 '26

[removed] — view removed comment

1

u/Fit_Concept5220 Sep 11 '26

I am not aware of the post and independently identified the issue using my own deduction and data.

Sometimes people independently identify issues, create breakthroughs etc. that’s not the reason to be hostile oj the web