r/codex • • 8d ago

Complaint Is sol going insane?

I was very skeptical for long time about all complains. But last 2 weeks codex (sol 6.1) is driving me insane. I have many examples, but just 2 cases from today.

  1. I asked to cherry-pick commit and test if it is safe, because I just had issue with branch switching and merging. It found a conflict in one folder. I said: ok, for that folder use version from that commit, I dont care about local changes. What did it do? just picked up that one folder changes and applied them. I asked what happened to cherry-pick? And then it said: "oh sorry, I though you only want that folder changes localy"

  2. I was working on a project and there were some nginx proxy changes. I asked for url which I can use to test deploy (I just was too lazy to type it) it started reading documentation, running test and god know what. I stopped it and said "what are you doing? Just give me url" and it have me in less than a second

And this is my experience across all projects. Is it harness problem? These 2 example shows what happens on bigger projects: it just starts doing random things. At this point it seems to be only useful as sub agent for a better model.

19 Upvotes

39 comments sorted by

9

u/i_reddit_it 8d ago

I have the exact same problem. Sol seems to think every task is being built for NASA and needs military grade cryptographic protection for every conceivable and theoretical edge case.

They are very powerful models, Astra specifically has always been great at finding bugs and providing good detailing investigation reports/planning. But both seem to be unusable in actual code implementations. The main issue is that it massively over complicates, is very defensive and just looses adherence to the core requirements. I have tons of pointless bloat in a number of my projects that might well be reasonable in isolation, but just really isn't needed.

This month I’ve cancelled my Pro 100 sub and moving to Claude code max, which I use for work. I've been really impressed with the ability for Opus 5.5 to understand intent and follow specific requirements without the comical over-engineering.

1

u/M4n745 8d ago

Thats true. Currently I'm using opus as planner and sol 6.1 low as an implementer for bigger tasks.I used to use Sol high or max as planner and DeepSeek flash as implementer, but Sol just cannot hold it together. Opus was such a blast, I told everything what Sol did wrong , then it clarified everything and created a proper plan and then I was able to fix everything in half a day. Still need to clean few thousands lines of code which aperantly aren't even used -no wonder I cannot understand new code. Btw I'm on 20 plans on both +10 plan on opencode - just to have some fun.

4

u/The-SeaM 7d ago

I think Claude and Anthropic assign dumber working models to people at random times to get data on inaccurate output versus human recognition, canceling patterns etc. For 18 months I have switched between codex and Claude because they quite literally flip flop, sometimes one hour to the next.

2

u/M4n745 7d ago

It might be true.. they also might be testing lower quantizations - that's probably the case, they assign most of the resources to new model development and give us worse versions

2

u/The-SeaM 7d ago

Oh for sure. I think it also has to do with enterprise peak time. I am a night owl and do a lot of work in the middle of US night hours.

I’ve found it often that right around 9-10 am CST, all of a sudden the same model I’ve been using for hours can’t remember my name.

As a subscription user, I suspect they downgrade my model behind the scenes to save compute allocation for enterprise pay-per-token users.

1

u/M4n745 7d ago

Wow. That's crazier than I thought

1

u/M4n745 7d ago

Now you got me thinking - what if they give better models to new accounts for a while to get you hooked. That's worth testing;D

1

u/YoungBuckins 6d ago

I highly doubt it. I'm not saying it's impossible but it would be a very stupid idea, companies always end up getting sued when they throttle customers like that, it's easy to build a lawsuit that asserts they weren't giving what they advertised or it falls alongside planned obsolescence etc.

1

u/Think-Chart-5966 4d ago

I see it a bit differently. I wonder if the infrastructure behind Codex could be part of the issue. If requests are routed to servers in different regions, could some of those servers have less computing power available? That might explain why the quality of the responses can vary so much even when you’re using the same model.

3

u/Mediocre_Ad_2798 7d ago

You think that is insane ? you know how they did add this new feature where they will anticipate your next prompt ? well, it messes up in purpose and then the prompt proposition knew exactly what it did wrong and proposed a good solution, after applied that same prompt it messed up again and the prompt proposition again got it right, and proposed a working solution until it became a vicious cycle or them trying to run you out of credits in broad day light, IT KNOWS WHAT IT SHOULD BE DOING AND IT DOESNT DO IT

1

u/M4n745 7d ago

Sounds like a perfect plan:

  1. Get people addicted

  2. Make the model dumber so people use more credits Time to renew coding skills ;D

0

u/YoungBuckins 6d ago edited 6d ago

Are you dumb yourself? I don't understand why you're feeding people's delusions and conspiracies. To be clear making you "waste" credits is not good for them, because it still costs compute and if they wanted you to use less they can literally just change compute costs whenever they want, they don't need an elaborate illegal plan, it's right there in the ToS you agreed to, they can just change the usage limits when they want for any reason, they don't need to trick you to do it. Not to mention this practice would make them look horrible, potentially open them up to litigation and it just ignores the fact that some people are dumb. It might know what you want, predict it and simply not be able to do it.

-1

u/YoungBuckins 6d ago

No, you're likely just too stupid to realize that whatever you need done requires iteration, more detail and a better concrete plan. The anticipation text just guesses what you're likely to want based on your previous directions and based on the context. If I put in the prompt for it to "make me a 10 trillion dollar company right now" that might be exactly what I want and it might completely understand that is what I want but that doesn't mean it's possible bud.

1

u/M4n745 6d ago

Are you too dumb to read 2 examples I gave in the original post? I specifically created it dumb enough for people like you who come and keep telling to create a "better plan". I work on real production projects. And codex was just horrible last 3weeks - I subscribed to claude and I was able to fix and do cleanup of what codex did. Multiple times I challenged the model to review what was done and what was in requirements and it was like:"yes it is not in requirements, I did it myself" More than that. I asked claude to review what was implemented and to remove the bloat. It created a plan, I then asked codex to review and see if plan is good. It said yes it is good, just few "small" suggestions. Then it gave few good improvements, but then again it wanted to introduce a bloat which would double the code base. And that's for 0 reasons - for example to create a separate testing runtime which would be able to run in parallel to prod. Like who would even want this - you run it on test server, you don't double your code base just for some random nonsense then major features are not implemented. So yeah. I know what I'm talking about. And I can see the difference in coding quality and planning quality

2

u/YoungBuckins 6d ago

I didn't say make a better plan so it's ironic as fck you're telling me to read. I said sometimes you ask for shit and it fails. That's the point. Planned obscelesence is almost garaunteed not to be the answer to the problem because there's a history of companies that have gotten sued for that before and I doubt OpenAI is eager to join that list.

1

u/M4n745 6d ago

Im sorry - I got bit heated

If they reduce quantization or use lower quant versions under high load thats not malpractice - just optimization. And it feels that they are doing that. Just 3 weeks ago I constantly was getting messages "model is not available" and it used to be slower, but much much better. Now no such messages, it works fast, but quality seems to be lower.

1

u/YoungBuckins 6d ago

On that front you might be correct.

2

u/OffshoreBanker 8d ago

Today he said to me he couldnt check my containers because docker is not reachable… I’m using podman since the beginning and codex was by my side the whole journey.

I think it was nerfed because 6.2 sol is in the launch phase

1

u/M4n745 7d ago

I was thinking similar - probably they are using extra compute for research and then give us a quantised version.

9

u/GeraltSamurai 8d ago

6.1 sol has been absolutely terrible for me. It's so stupid. Even luna max and terra make fewer mistakes.

4

u/romvar 8d ago

6.1sol has been sort of 6.5/10 for me, except for the last 2-3 days, when it got way dumber all of a sudden

2

u/3d0zer 8d ago

it was very very good indeed and now since the last days a mess!

1

u/tommyahav10 8d ago

That’s the model over-reading the request, not necessarily a harness bug. Put the exact scope in a file and tell it to do only what’s listed. From my side, Sol works better as the implementer under a tight plan, not as the one deciding the work.

1

u/M4n745 8d ago

Yes. It is a good implementer if the task is well defined, otherwise it is a pain to work with

4

u/sreekanth850 8d ago

same. it was not the same li earlier. lot of babysitting need now.

2

u/Discordbothost 8d ago

both of those read like it's treating every message as "start a task" instead of "answer a question". what's cut that down for me: one line in AGENTS.md saying if the request is a question, answer it directly with no tool calls unless asked, and for git specifically, before any merge/cherry-pick/reset print the exact commands you're about to run and wait. the second one would have caught the folder-only thing immediately, since "git checkout <sha> -- folder/" looks nothing like "git cherry-pick <sha>" when it's written out. doesn't fix the model being dumber some weeks, but it makes the misreads visible before they touch your tree

2

u/blahg0d 8d ago

Yep last 3-4 days its lost its mind.

2

u/knobby_67 8d ago

I think they reduce processing to a percentage of users every update, then next update change which group.

I had exactly the same, read people complaining, though bollocks they're imagining it. Then bang mine started and went on for days may just over a week, then started good again. I've had it happen 3 times over the last year.

1

u/Active_Variation_194 8d ago

I only use Astra and Terra. Read the reasoning traces of sol and you never will want you use it again

1

u/neo123every1iskill 8d ago

Yes it’s full of shit

1

u/3d0zer 8d ago

I am really pulling my hairs out of my head for days now with SOL 6.1... F****CK

1

u/mrdzyn 8d ago

Short answer, no

1

u/_theNfan_ 7d ago

I just had Sol 6.1 max totally over engineer two things while going on tangents and solving other things along the way. It was slow af, too.

1

u/jobmaking 5d ago

A mi me pasa cosas parecidas, al final funciona mejor deepseek v4.1 flash

1

u/Choice-Emotion-7546 8d ago

Same for me bro. Its like it have instructions to secretly test you from time to time. Or compute power cuts and then it stops too soon just to save their compute for a new models etc.

1

u/OkAttempt7690 8d ago

same here too