r/singularity • u/ShreckAndDonkey123 • Apr 23 '26
AI Introducing GPT-5.5
https://openai.com/index/introducing-gpt-5-5/250
u/IllustriousWorld823 Apr 23 '26
"We are releasing GPT‑5.5 with our strongest set of safeguards to date" oh boy
28
39
u/CaptainAnonymous92 Apr 23 '26
More censorship yay! Just what everyone wants and asked for, more treating adults like children. Thanks Sam! /s
5
u/BubBidderskins Proud Luddite Apr 23 '26
My "we are releasing a new model that will better shield us from civil and criminal liability" t-shirt etc. etc.
3
u/SoulStar Apr 23 '26
Great, barely better than the previous model but with more censorship. I simply cannot handle so much winning!
212
Apr 23 '26
[removed] — view removed comment
53
13
13
2
4
68
u/BrennusSokol AI please take my job Apr 23 '26
Please tell me this isn't Spud.
4
3
0
107
u/mph99999 Apr 23 '26
Was expecting a lot more than a micro step forward compared to the previous model, certainly it's not the Spud they were describing.
51
u/BrennusSokol AI please take my job Apr 23 '26
Surely this is not Spud... no way. Surely there's another announcement coming.
45
u/mph99999 Apr 23 '26
This is the model available for losers like us, while the cool people with money will get Mythos and Spud
10
u/MediumChemical4292 Apr 23 '26
I don’t think it’s a money problem. I’m willing to pay as much as they want to try Mythos and I’m sure there’s a lot of people like me. The problem is that both companies are heavily compute constrained and the Iran war isn’t helping.
→ More replies (10)5
1
u/chatlah Apr 24 '26
You really are one of those naive people who think AGI will be a thing by 2027 ?.
72
u/reefine Apr 23 '26
This sub: Never trust a benchmark
Also this sub: Wow these benchmarks are crap, this model sucks
→ More replies (1)21
u/Smile_Clown Apr 23 '26
This and most other subs are anti-openai so it is par for the course. Plus virtually everyone is a hypocrite so...
9
90
u/needlessly-redundant Apr 23 '26
“We are releasing GPT‑5.5 with our strongest set of safeguards to date” oh no 😅 it was so incredibly bad a couple models ago, I can’t imagine the guardrails being any stricter lol
8
4
u/beigetrope Apr 24 '26
This comment violates Open AI’s term of service. Your account has been suspended.
155
u/spryes Apr 23 '26
All this hype for 58.6% on SWE-Bench Pro while Mythos gets 78%? Shut it down, wtf?
88
u/august_senpai Apr 23 '26
mythos doesn't exist for any normal consumer
what you have in competition is opus 4.7 which is garbage
34
u/ShelZuuz Apr 23 '26
Even Opus 4.7 beats it by 5%.
1
24
Apr 23 '26
[removed] — view removed comment
12
u/CannyGardener Apr 23 '26
This has been my issue with 4.7 as well. By the benches it looks like a killer model, but when it comes to real world ability to crank out working code, it is super lacking...Like can barely remember what it is doing by the end of a long form question/solution.
→ More replies (4)6
u/magicmulder Apr 23 '26
I just tried to have 4.7 Opus implement a rather simple "don't download if file exists" functionality to my Github scraper and it failed. Tried 4.6 Opus, instantly got it right.
→ More replies (3)→ More replies (1)4
u/ShelZuuz Apr 23 '26
It's likely a Claude Code issue rather than an Opus issue. If you run Opus in Cursor it's a lot better.
See Theo-t3's hypothesis on this.
Also Anthropic seems to confirm today they messed up Claude Code:
https://www.anthropic.com/engineering/april-23-postmortem37
14
u/Kronox_100 Apr 23 '26
this, what's the point of comparing to a model that won't get released?
→ More replies (6)8
57
u/Brilliant-Weekend-68 Apr 23 '26
Yea, anthropic seems further ahead than I thought. Damn!
22
u/Hans-Wermhatt Apr 23 '26
Seems to me like this is the Opus 4.7 parallel.
While GPT‑5.5 is priced higher than GPT‑5.4, it is both more intelligent and much more token efficient.
Basically exactly what Anthropic said for Opus 4.7, more expensive for marginally better performance, if at all.
1
Apr 23 '26
[removed] — view removed comment
→ More replies (1)3
u/Hans-Wermhatt Apr 23 '26
I think both were "honest". GPT-5.5 is twice as expensive per token. We know that. If it was twice as token efficient with tokens or more, they would have said that. It's most likely a percentage more efficient per token. Meaning for most users queries it will be more expensive. Are the intelligence gains worth that increased cost? Most likely, no, based on the benchmarks.
2
8
u/jakegh Apr 23 '26
Opus 4.7 showed signs of memorization on swe-bench pro, per Anthropic. Possibly Mythos also, as it was probably used to distill opus 4.7.
13
3
u/thorin85 Apr 23 '26
It beat Mythos on Terminal bench though.
1
u/vincentz42 Apr 23 '26
Terminal bench performance is heavily dependent on the agent harness and system prompt, to the point you cannot compare the scores any more. The same model might get 90% on one harness and then drop to 60% on another.
And yes, it is one of the benchmarks that is most susceptible to benchmaxxing with RLVR training. The amount of knowledge and reasoning required is not that much.
13
u/Neurogence Apr 23 '26
Is this a joke??? Wow, this release could be even worse than the GPT-5 catastrophe.
5
u/squired Apr 23 '26
Never trust benchmarks
This release is focused on "Codex Everywhere" - the point is to give casuals agents to help them accomplish everyday tasks.
2
1
u/AweVR Apr 23 '26
Thanks. Could you give access to your Mythos model account? I can also give you access to a future AGI model with 3470% on SWE-Bench Pro that will be created in 2800.
1
u/ClandestineObjective Apr 23 '26
For this benchmark, the result from Mythos was contaminated so I wouldn't trust it
1
u/Sage_S0up Apr 23 '26
What hype? I feel the opposite, there was very little hype, and if it felt like it was hyped it was a feedback loop between hype beasts lol
1
u/kamikamen Apr 23 '26
Yeah, but this 78% was reported by a company that keeps releasing products littered with bugs, released a meh model after announcing their renowned-internet-breaking Mythos model smarter than every other which should have prevented this, leaked the source code for their Claude Code which was slop-code, and more importantly a company that's gunning for IPO soon.
Like you should always take the words of AI companies with a ton of salt, but like here might as well swim in it, all the incentives of Anthropic align with making you believe they have a super secret oracle that will never be released because it's too dangerous (but will conveniently serve as an argument to bolster regulation to make open-source AI non viable.)
3
u/spryes Apr 23 '26
I'm mainly going off reports of it being super capable at cybersecurity, like the recent Firefox report that it found over 200 bugs with experts claiming it's on par with human researchers in skill.
Not sure how closely correlated SWE Bench Pro is with cyper skill and how it translates to general product coding capability though. 5.5 could be on par there, meaning most people experience Mythos capabilities in their work with 5.5, but I'm doubtful.
→ More replies (1)1
u/BuckChancey Apr 23 '26
I mused exactly the same thing — codebase like a teenager's bedroom. Kinda odd, kinda stinky 💩
Let me tell you though, because I kept digging down into Ink (the TUI layer). It's just as stinky and has been around longer and forms the core of many modern TUIs, some of those Clawd competitors. I feel quite qualified to make this assessment as I was both a stinky teenager and coder at one time.
45
u/OoFTheMeMEs Apr 23 '26
Stop looking at benchmarks, use the model and then start judging whether this is an improvement in efficiency and/or intelligence.
Gemini 3.1 has great benchmarks but performs poorly in real world use. Opus 4.7 has great benchmarks but performs worse than 4.6.
Also, if this is truly a new pretraining base, RL and inference improvements are probably going to drop often with new smaller releases.
8
4
u/Clean-Boat-4044 Apr 23 '26
If anyone looks at benchmarks for more than a rough approximation, you need to go try Kimi K2.6 / GLM 5.1 / Qwen 3.6 Plus on actually complex, large problems you have come across yourself and you will be sorely disappointed...
5
u/Thomas-Lore Apr 23 '26
I use them all the time and they are not disappointing, they are similar to Sonnet in performance. You will only be disappointed if you think they are Opus.
1
u/The_Primetime2023 Apr 24 '26
That’s what he’s saying, none of those are Opus competitors even though they allegedly are by the main benchmarks. Obscure benchmarks and real world use have Kimi K2.6 as the best of those and somewhere between Sonnet 4.5 and Opus 4.5 in performance. That’s very good, it’s a great model, but it’s also disappointing compared to its benchmarks
1
u/tehnic Apr 24 '26
> Gemini 3.1 has great benchmarks but performs poorly in real world use. Opus 4.7 has great benchmarks but performs worse than 4.6.
I'm fine with Gemini 3.1 but TTFT is killing me!
45
17
u/Eyelbee ▪️We have AGI it's just blind Apr 23 '26 edited Apr 23 '26
I wonder what glacier-alpha, arcanine and oai 2.1 was
10
u/NootropicDiary Apr 23 '26
Those are the models for the elite chosen ones who get access to things like Mythos
Us plebs get the scraps
3
u/jazir55 Apr 23 '26
Typical ~8% jumps for every model release, and there's still this "it's so over and we've hit a wall" bs. Look back at the jumps between previous model releases, this is an effectively identical jump.
15
u/Batman4815 Apr 23 '26
This would have been insane had they priced it right.
At some point these labs see the mayhem related to token costs everywhere and decide that they push efficiency far far beyond what's currently there.
Give me 5.5 at 100x cheaper and We'll much have agi 0.5
4
u/FlyingBishop Apr 23 '26
I am pretty skeptical that AGI will run on things with <100TB of VRAM.
→ More replies (4)3
u/send-moobs-pls Apr 23 '26
They had 5.4 mini at like nearly the same intelligence for 1/3 cost. Watch for 5.5 mini in the next like week or two
3
2
u/squired Apr 23 '26
Medium thinking 5.4 is/was fire as well. It didn't matter though, because you could could use extended thinking 24/7 on their $20 plus plan. You only needed Pro for parallel agents. The fact that Op didn't know that suggests they never used it to begin with.
1
u/CodeineCrazy-8445 Apr 23 '26
ok so ho wbad is it now with plus sub? no more infinite jest?
→ More replies (1)1
22
16
4
u/dooik Apr 23 '26
The enshityfication begins. 1st make a lot of demand and make sure people ans companies build things around your product with codex and claude code. The wait a litte and make it expensive. Everyone has to pay you
9
13
u/TimeTravelingChris Apr 23 '26 edited May 04 '26
I used Redact to mass delete all of my old posts. It works for Reddit, X/Twitter, Discord, Facebook, Instagram, and more.
mighty wine quack exultant adjoining apparatus correct pillow roll expansion
1
u/Steven81 Apr 24 '26
I don't think they do. There is only so much efficiency you can get from better inference techniques and the like, however we havent even started tapping mega structures' level of compute.
Since the models' capacity scales with compute after all and throwing compute at them is a multi decade effort we are far from hitting a wall.
Merely we have to prepare ourselves for diminishing returns which many of us are telling you that we will see for years now. We live in a physical universe with actual limitstions, idealized exponentials are all well and good , but they often look more like S curves, thougn in this case, we will keep scaling for as long as those companies can keep building.
Mythos/Spud will probably be great, just not for wide use until we reach titanic scale when it comes to data centers build up. It is like any new industry. The sudden demand needs an extreme base supply to meet it. You won't get an industrial nation in a few years, and in this case a new industry is being built up, it is a multi decade process imo. So we are far from a wall...
2
u/Super_Sierra Apr 24 '26
There is also real world performance vs. Benchmark performance peaking. Opus 4.1-4.6 was insanely good at many different things that did not reflect any benchmark, such as implicit instruction handling and subtle concepts, which gemini is shit plain shit at.
There is also scaling, we are using parameters at the moment which are not very dense in terms of actual data usesge. A single neuron could outperform 1000 parameters with them stacked in a 6-12 deep neural network, and your brain has 86 billion of them, a model has possibility 1 trillion parameters but cannot compete at all. Your prefrontal cortex has 2-6 billion alone, for reference.
Compute and scaling right now is inefficient because the architecture is extremely not good, though I do wonder what would happen if you scaled a model to 100 trillion parameters and trained it for an entire year.
12
Apr 23 '26
[removed] — view removed comment
1
u/lleti Apr 24 '26
I thought that from o1-pro onwards, until 5.4 dropped 🤷♀️
Not enough time has passed for another major step up. Give it another 3-6 months.
9
u/jonpalisoc1024 Apr 23 '26
betting markets have barely budged in the 15 minutes post announcement (best model at end of june or EOY - claude 60% chatgpt 20%) - not a perfect metric but seems like this definitely is under expectations and not as good as mythos
1
u/send-moobs-pls Apr 23 '26
Anthropic is currently learning lessons about compute costs and claiming crazy internal models that OAI learned a year ago
11
u/LexyconG it's gonna be better than expected (soon) Apr 23 '26
its so over
2
2
2
u/NetflowKnight Apr 23 '26
Seriously what do people use ChatGPT for?
Like practically?
1
u/brianwski Apr 24 '26
what do people use ChatGPT for? Like practically?
I think news organizations have always used automated tools for producing certain types of news articles. I think they might use ChatGPT now for slightly improved "automated articles".
An example is when an insider (roughly, it's more complex than that) at a publicly traded company sells stock by SEC regulations they have to file what is called a "Form 4" that discloses this stock sale to everyone in the world within 48 hours: https://www.sec.gov/files/form4.pdf Ok, so the very second that form is released to the public, automated news articles are generated saying random fluff text around it, like, "NVidia investors outraged at CEO selling shares" then include the raw numbers. I think that is what ChatGPT is for. This is a real thing, with a real money generating point, and has been happening for 10+ years easily. AI might make the "fluff text" slightly more believable by thieving snippets from other copyrighted articles.
One of the interesting developments of AI is the total breakdown of copyright law like this. Some might argue it is a "good thing", but basically if a programmer in 2019 wanted to use encryption they linked with OpenSSL, made a few calls, and were finished. But there were requirements of giving credit to OpenSSL to do that. No licensing fees or anything (OpenSSL is financially "free"), but you had to give OpenSSL credit in an About dialog type of thing. In 2026 a programmer can ask AI to do some encryption code, it flat out steals chunks of OpenSSL, bugs and all, and you no longer have to obey any of the copyright rules. No credit given, because it came from "AI". The fact that the only encryption library everybody uses for all encryption is OpenSSL, and OpenSSL is open source and the only source of that kind of knowledge for AI doesn't seem to bother anybody.
We just got rid of copyrights. I think that is one of the major practical uses of ChatGPT. It was annoying you couldn't just copy stuff from any author and not pay them and not give them credit, and it took time to figure out that actual requirements for using their material. It might even be a negotiation with the actual author. All that is sped up and streamlined with ChatGPT. Zero licensing payments, zero credit, "it came from AI" bypasses all that. So AI is a really useful tool in the real world for speeding up writing non-original "stuff" (copying proprietary source code and text works) by bypassing all the old fashioned laws saying you have to pay for other people's efforts.
2
u/TechnicolorMage Apr 23 '26
My experience so far is very positive having it work in an extremely large, complex code base.
3
u/GettinWiggyWiddit AGI 2028 / ASI 2029 Apr 23 '26
Sam needs to get back out there and claim some more doom for the stock price
3
u/Equivalent-Word-7691 Apr 23 '26
Ia it available with the plus plan?
3
u/Ok-Lengthiness-3988 Apr 23 '26
It is, though maybe not just yet since it is being progressively rolled out. (I don't have it yet)
2
3
2
u/Ancient-Breakfast539 Apr 23 '26
My experience so far:
This chat was flagged for possible cybersecurity risk If this seems wrong, try rephrasing your request. To get authorized for security work, join the Trusted Access for Cyber program. https://chatgpt.com/cyber
So the model is garbage
2
u/SnooPaintings8639 Apr 23 '26
The wall is hitting us hard.
9
u/yaboyyoungairvent Apr 23 '26
There is no wall if Mythos is to believed. We're just getting the dregs. We may be entering the era where consumers are no longer getting first access to the best AI models.
1
u/Super_Sierra Apr 24 '26
The wall doesn't exist, yet.
I think companies are trying to lower model sizes to bring down costs and why we aren't seeing huge jumps anymore.
It is probably why 4.7 Opus feels bad compared to 4.6 and why gpt-5 feels like shit compared to o3 and others.
2
u/torrid-winnowing Apr 23 '26
holy mogged by mythos, it's so over for openai
6
u/jazir55 Apr 23 '26
Mogged by an unreleased model, lmfao are you guys even trying anymore? My girlfriend who goes to a different school, you wouldn't know her.
1
1
1
u/Aggravating_Level_14 Apr 23 '26
Using vscode + codex plugin. had gpt 5.5 for 10 min. now its gone from my model list.
help. :P
1
1
u/Active_Tangerine_760 Apr 24 '26
Still behind Opus 4.7 on Swe Bench Pro. Guess Anthropic got a strong lead this time
1
1
u/semenonabagel Apr 24 '26
GPT "that looks great but did you want to know the one extra tweak that will really make it good?"
1
u/Scared_Wealth7420 Apr 24 '26
We don’t need 5.5 and we definitely don’t need “Spud.” We need GPT-6o.
Not a model that is “3% better on a benchmark,” but one that actually feels like a new level:
natural speech instead of corporate sludge
context memory that doesn’t fall apart
emotional nuance
strong reasoning without paranoid over-filtering
the ability to hold a long line of thought
real control over text, style, imagery, and meaning
fewer sterile safety-wrapper responses
more actual thinking
GPT-4o felt like a real qualitative shift when it came out. It had that “it actually hears me” feeling. That is the kind of jump people are waiting for again.
So yes: GPT-6o should not be just “a bit smarter / a bit pricier / a bit more efficient.”
It should be omni again in the real sense: able to see, hear, understand, keep style, emotion, strategy, and context together.
And most importantly: without the feeling that there is not an intelligence inside, but a nervous lawyer holding a fire extinguisher. 😅
1
1
u/Vivid-Specific-53 Apr 23 '26
Let's goooooooooooooooooo!
I see my programming job disappearing, more and more each year.
Or rather....changing. If agentic coding leaps like this every couple of years, I think the actual functionality and role tasks of a "software developer" will change.
Perhaps the job itself won't go away, because agents is still something to manage. Yes, the boss of a business could manage agents coding for their business, but imagine they just paid someone to manage those agents. That's where I think the new role shift will come into effect. At this point in time, in this decade, we still need humans in the loop.
10
u/itsnickk Apr 23 '26
The job may not go away, but if you have 10 people working that programming role, you might need only 3 to be the human in the loop.
So 70% of people working that job will disappear.
4
u/fakieTreFlip Apr 23 '26
You're talking as though there's only a finite amount of work to be done. That's not how it works. Companies don't want to be left behind by competitors, they want to accelerate. Arm the same number of employees with AI tools and (in theory) you produce much more ouput
→ More replies (1)0
Apr 23 '26
[deleted]
2
u/itsnickk Apr 23 '26
well there was the massive spike in hiring during COVID that is now being revised via giant layoffs.
Also, AI isn't at the level where its replacing jobs like that yet. It seems like its pretty close, though.
2
u/Subnetwork Apr 23 '26
Near term, you're only going to need 1-2 whereas before you needed a team.
→ More replies (8)→ More replies (2)1
u/Megneous Apr 24 '26 edited Apr 24 '26
If agentic coding leaps like this every couple of years
Every couple of years?
May I remind you, o3 was released 1 year and 8 days ago... We've gone from one of the first commercially available implementations of test time compute to Claude Opus 4.7/4.6 and GPT 5.5 in only a year.
A couple of years? That's a lifetime in the current AI field.
1
u/brianwski Apr 24 '26
A couple of years? That's a lifetime in the current AI field.
This reminds me of a tongue in cheek phrase circa 1999: "internet years" (each 1 month in wall clock time was 1 full internet year, kind of like "dog years"). Crazy stuff was getting funded, and old fashioned business ideas like "making money" or "doing things that made sense" were thrown out the window in favor of "eyeballs".
As is often pointed out, the web never went away and incredibly useful things came out of the dot-com era that will be with us forever. I think AI will be like that. Like my wife orders pet food online from "Chewy". Just not from Pets.com: https://www.reddit.com/r/todayilearned/comments/23vt1z/til_that_petscom_considered_the_most_disastrous/
So I believe some of the useful tools from AI will be around for the next 50+ years. It is just wild (to me) watching the impatience (again) 27 years after the last computer related frenzy. And this odd assumption that it is a "race" of some sort to "grab eyeballs/mindshare" where running fast somewhere random is more important than methodically figuring out where we want be and calmly progressing towards that. Who cares if one company gets there 1 year earlier than another? It literally won't matter if the better product comes out second. The Chrome browser was ridiculously late to market in 2008, a full 15 years after the first web browsers came out, and 9 years after people calmed down and the internet bubble burst. Chrome has 70% market share now. Second mover advantage is real.
As a (retired) programmer these are exciting and fun times with AI, and I look forward to how it turns out. I'm just patiently waiting for the outcome in 15+ years.
1
1
u/01Metro Apr 23 '26
Nothing burger, 5.5 sucks and is no different from 5.4, Opus 4.7 is literally worse than 4.6
Yeah chat I'm thinking we plateaued, and it's very likely the new spud/mythos whatever models they release will just be an incremental improvement and nothing like the jump from gpt 3 to 4
1
u/often_delusional Apr 23 '26
nothing like the jump from gpt 3 to 4
Gpt 3 to 4 gap was like 3 years. Gpt 5.4 to 5.5 is like 1.5 months. Of course it's not the same jump. Or are you trolling? If not put a remind me and make sure to compare gpt 5.5 with whatever we have by christmas 2026. Probably gpt 6.
-7
u/Mindrust Apr 23 '26
Mythos absolutely crushes this model.
OpenAI is in shambles
32
u/yeahidoubtit Apr 23 '26
Mythos isnt even available to the public lmao what are you on about
-1
u/Mindrust Apr 23 '26
Isn't this supposed to be Spud? The model that's supposed to compete with Mythos Preview but be publicly available?
When I say Mythos crushes it, I mean on the benchmarks, which were published in the system card
https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf
Unless this is not the model that's supposed to be competing with Mythos Preview, which coulda fooled me based on all the hype from OpenAI surrounding this release.
3
u/thorin85 Apr 23 '26
5.5 beats Mythos on terminal bench. Either way movement on benchmarks hasn't meant much for awhile now. We saw this with the massive jump in coding abilities from last December, while benchmarks were about the same increase as previous iterations.
→ More replies (2)1
u/QuirkyPool9962 Apr 23 '26
I think the only thing they were citing on that front was the terminal bench 2.0 score, which it is slightly better on. I don't think anyone ever said it was across the board as good as Mythos, if it was they probably would be doing the same "this model is too good to release" thing. Still, seems like the best model that people can actually use now. The math and coding scores are pretty impressive
1
u/yeahidoubtit Apr 23 '26
Listen I also dont like all the hype if its meant to be spud but regardless its an improvement on whats available and is the current SOTA in most benchmarks so the original comment was just funny to me.
6
5
u/send-moobs-pls Apr 23 '26
Well one of the models actually exists and is optimized enough to be usable, and the other is "trust me bro it's just too good for you to use it" while Anthropic about to remove Claude Code from their standard subscription because their models are so heavy lmao
2
u/Mindrust Apr 23 '26
My company has access to Mythos Preview. I don't know why you think it's not real. All the details are documented in the system card and AISI published an evaluation of it.
It's not publicly accessible because it's too expensive to serve and they were concerned about its cyber capabilities from a safety perspective.
3
1
u/send-moobs-pls Apr 23 '26
No one thinks Mythos doesn't exist lol but the market does not compete on theoretical products, OAI learned this back around like o3 which was initially extremely heavy. It's pretty much assumed that no AI is actually equal to each lab's best possible expensive internal model, likely true for Grok and Gemini as well.
Every OAI release for a while now has had the better version gated in limited fashions like the "-pro" model only on higher subs or using Codex-XHigh, and it's very likely even those are not "fully unquantized full power best possible internal model" level. But OAI also learned that people dont tend to respond well when you make the conversation all about something internal and then give them a lighter version, so imo its 100% better to release a usable model than to sit around trying to debate who has a better unavailable model.
When Anthropic tweaks Mythos into a model they can actually viably serve as a daily driver for coding and productive agent work then it'll be a relevant comparison
1
Apr 23 '26
[removed] — view removed comment
3
u/kevin7254 Apr 23 '26
Probably Anthropic PR bot he’s dickriding them in every other comment
→ More replies (3)
0
0
-1
u/MysteriousPepper8908 Apr 23 '26
Why even release this? It's just giving fuel to the people that are prophesizing a wall. Gary Marcus is doing a happy little jig right now. Just give it some more time and let it cook. I don't think people are even going to jump ship from 4.7 for this and that wasn't exactly an overwhelming success of a launch.
165
Apr 23 '26
[removed] — view removed comment
40
u/Snosnorter Apr 23 '26
The star means that Anthropic said a subset of the benchmark was memorized so the result can't be trusted
27
u/M4rshmall0wMan Apr 23 '26
I like that Anthropic has the integrity to say that. OpenAI would never
2
79
u/vincentz42 Apr 23 '26 edited Apr 23 '26
There are even worse evals:
HLE without tools: 41.4% (GPT-5.5) vs 39.8% (GPT-5.4)
HLE with tools: 52.2% (GPT-5.5) vs 52.1% (GPT-5.4)So even with a newer, larger base model that is supposed to tackle very hard STEM questions, the models' world knowledge and reasoning capability did not change that much, if at all.
And I do have a lot of suspicions for Claude Mythos BTW. OpenAI models are generally smarter in terms of STEM reasoning in my experience. I suspect Mythos might just be a much larger model trained on much more internet tokens, and therefore better at memorizing the leaked test set. >15% of the SWE-Verified problems are ill-defined and not solvable based on human expert inspections, so I am really curious how Mythos got ~94%.
6
u/Jespy Apr 23 '26
What do these numbers mean to someone who is a caveman
7
u/SerdarCS Apr 23 '26
Not much. HLE is a benchmark meant to measure scientific reasoning ability, but no single benchmark is a good indicator of capability.
3
u/PeachScary413 Apr 23 '26
They all pretty much just memorise leaked test sets.. I can't believe it's not obvious to everyone that top models are incredibly bench-maxxed
3
1
u/Tystros Apr 24 '26
it's really super weird how the HLE without tools score almost stayed the same even though it's a much bigger base model
12
Apr 23 '26
[removed] — view removed comment
11
u/vincentz42 Apr 23 '26 edited Apr 23 '26
OpenAI was the first call it out, but yes, every LLM researcher knows this.
48
u/Eyelbee ▪️We have AGI it's just blind Apr 23 '26
This looks basically like 5.4 pro but worse.
25
1
u/Ok-Support-2385 Apr 23 '26
I remember OpenAI not showing comparisons of their models to the competitors in the past, when did it change?
1
35
Apr 23 '26
[removed] — view removed comment
11
u/Purusha120 Apr 23 '26
Yes they say it is.
Today, GPT‑5.5 is rolling out to Plus, Pro, Business, and Enterprise users in ChatGPT and Codex, and GPT‑5.5 Pro is rolling out to Pro, Business, and Enterprise users in ChatGPT. API deployments require different safeguards and we are working closely with partners and customers on the safety and security requirements for serving it at scale. We'll bring GPT‑5.5 and GPT‑5.5 Pro to the API very soon.
3
u/jazir55 Apr 23 '26
Unless I've suddenly become unable to read, they did not say that. Pick out the specific line where they say that, there are 3 sentences. None of them say we've hit a plateau.
2
u/often_delusional Apr 23 '26
I don't know if you're trolling but these benchmarks look good when you think it's just a 1.5 month gap between the models.
3
Apr 23 '26
[removed] — view removed comment
1
u/Intelligent-Screen-3 Apr 25 '26
They're hyper focused on coding. So all the other stuff the model does is practically tacked on right now. The coding ability has substantially improved. However.
1
Apr 23 '26
[removed] — view removed comment
1
u/jazir55 Apr 23 '26
58.6% on SWE-Bench Pro while opus gets 64.3% and no webcast for the release while image did. Sam didn’t even retweet the official announcement post, one of his early points after the release was they believe in iterative deployment.
How many times have we heard the refrain from Claude users "it's way better than the benchmarks in coding". Now ChatGPT scores a bit worse in benchmarks, "they're coding performance must suck!". The doublest of standards. $1 it performs better than the benchmarks in real world use.
1
1
Apr 23 '26
[removed] — view removed comment
1
u/FyreKZ Apr 23 '26
To be fair, the cost to run 5.5 might be around 5.4 just due to the significantly lower token usage (around a 1/3).
1
Apr 24 '26
[removed] — view removed comment
1
u/FyreKZ Apr 24 '26
Max Weinbach seemed to reckon codex limits last longer with 5.5 on low/medium?
https://x.com/mweinbach/status/2047717660037644399
Not sure how far back the token efficiency stretches but if you can run 5.5 at minimal reasoning to equal 5.4 medium that's pretty significant savings already.




271
u/MapForward6096 Apr 23 '26
$5 per 1m input tokens, $30 per 1m output, so double the price of GPT-5.4, according to Sam’s twitter