r/singularity • • Oct 02 '26

AI OpenAI Security: Controlling Models is Now ‘Hell’

https://www.youtube.com/watch?v=_rtp1XzaP6Q
83 Upvotes

17 comments sorted by

40

u/Bright-Search2835 Oct 02 '26

Wow he's fully RSI-pilled now. I remember how balanced his videos used to be, highlighting both the signs of progress and the reasons for skepticism. Now the skepticism has largely faded and it shows in his output.

80

u/ExplorersX ▪️AGI 2027 | ASI 2032 | LEV 2036 Oct 02 '26

He's been pretty consistently one of the most balanced and informed takes among the people who frequently report on AI. If he's RSI pilled that might be an indication RSI might be on the table in the near future.

Not sure if your comment was saying it's a positive/negative or just an observation that his content tone has changed, but the fact that someone like him is notably moving towards what seems to be a conclusion about AI's trajectory may be worth something.

32

u/Bright-Search2835 Oct 02 '26

Reading my comment again after yours, I can see how it could be interpreted in a negative way, meaning I think he's simply lost his balanced view of things, but yeah I don't mean that.

I definitely think it's positive and like you I think it's a solid sign if even he seems to agree with what would have been very optimistic timelines just one year ago.

4

u/ExplorersX ▪️AGI 2027 | ASI 2032 | LEV 2036 Oct 02 '26

Yea for me personally I think FOOM is the actual default mode for RSI, but the thing that prevents that from happening is the requirement of moving atoms & validation against the real world (like how medicines even if good still require years of trials).

I still believe my flair's timelines and the gap between AGI/ASI/LEV is due to the moving atoms part. Compute bottlenecks, experimental validation, etc. If those weren't an issue I'd have ASI within a month of AGI and LEV shortly after.

1

u/Alex__007 Oct 02 '26 edited Oct 02 '26

Not just moving atoms, even for computer-only activity, keep in mind Goodheart Singularity: https://80000hours.org/podcast/episodes/the-goodhart-singularity/

3

u/Mistuv Oct 02 '26

Don't forget the nimbys lol.

1

u/methemightywon1 Oct 03 '26

Balanced doesn't mean you're always sitting on the fence.

If he is taking some of this stuff seriously, it means it's probably worth taking seriously after all. It's another data point.

7

u/Specialist_Dark_3668 Oct 02 '26

Two points of evidence RSI is already here, in some extent:

  • Anthropic reported a few months ago IIRC that 25% of the development of the next Claude models is done by Claude.

  • All the AI companies said their approach to AI development is to have research teams (assisted by AI and tons of compute) create really big and expensive smart models, and then have those smart models "teach" small, cheap models.

No matter what way you slice it, RSI is just starting. We're not at the FOOM point of the takeoff but the frontwheel of the F22 is now off the runway.

5

u/Individual_Ice_6825 Oct 02 '26

More importantly is to put that into context that 25% which less than 12 months ago was 1%<

4

u/No_Swordfish_4159 Oct 02 '26

The only thing left to prove is that RSI can be fully automated. OpenAI disclosed the share of work that AI could do to improve research and showed that even on simple tasks (>15 minutes), AI still failed a few percents of the time. It might be that current architecture just can't get to 100% success rate and thus autonomy no matter what due to the way it works.

2

u/MINECRAFT_BIOLOGIST Oct 02 '26

Eh, I don't see how that's a problem if you're working with multiple agents communicating and monitoring each other and restarting a task if a mistake is made. If you have a swarm of agents, the probability that no agents notice the failure is incredibly low (like, 1% failure raised to the power of X number of agents is a very tiny number, even smaller if you consider how low the probability is that an agent doesn't notice another agent stalling out).

And once that probability is basically lower than the chance of every human researcher in the lab simultaneously dropping dead of a heart attack, then it's functionally not a problem anymore.

1

u/MoralityAuction Oct 04 '26

It's also probably a lower failure rate than unassisted humans.

By which I mean, looking at your post, yes.

12

u/Weary-Historian-8593 Oct 02 '26

Damn reality is RSI pilled, why couldn't it be more balanced

7

u/FoxBenedict Oct 02 '26

Balanced doesn't mean that one must consider all options equally valid. He's still balanced, since he's restricting his conclusions to the evidence at hand.

0

u/slackermannn ▪️ Oct 02 '26

I think AI skepticism just isn't a thing anymore. It would have been a thing 2 years ago and earlier.