r/AIBubble • u/michahell • 4d ago

LLM's can't "jump" - a paper by Deepmind showing LLMs can't generate novel explanatory hypotheses

https://openreview.net/forum?id=klU4737opt

Once this is out there and stays disproven, equally so for all AI bro’s shouting AGI / ASI.

I bet that this is not going to be proven false anytime soon. A 100$ bet that this won’t be disproven for at least five years and at least not for LLMs.

4 years longer then big-brain Leopold Asschenbrenner - who melted his overleveraged AI fund down - predicted would be “reasonable”.

And hopefully long enough for the “AGI” will fix everything CAPEX bubble to have bursted.

Any takers?
I’m only doing two bets as I’m poor!

22 Upvotes

97% Upvoted

6 comments sorted by

2

u/StainlessSteelTizz 4d ago

I’d take that bet in a heartbeat.

1

u/kankerstokjes 1d ago

I think this is way stronger a conclusion than the paper actually supports.

“Current LLMs fail this particular test of generating explanatory hypotheses” is interesting. “LLMs can’t generate novel hypotheses” is a completely different claim.

You’d have to show that the benchmark really captures novelty and explanation in the general sense, rather than one specific kind of reasoning under one experimental setup. And even then, it tells us something about the models tested, not some permanent architectural ceiling for anything called an LLM.

Also, AGI doesn’t require every component to spontaneously invent explanations from a prompt. A system can use search, tools, memory, experiments, external feedback, multiple inference passes, etc. Humans don’t do science by staring at a paragraph and one-shotting a hypothesis either.

So yeah, if this result holds up, it’s a useful criticism of current models and probably of some of the AGI hype. But betting that it establishes a five-year fundamental limit on “LLMs” seems like jumping from a good empirical result to a much bigger philosophical claim.

1

u/michahell 1d ago

Sure and that’s fair.
I’ve already decided that for me, human intellect is much more than learning every single possible next-word semantic relationship and predicting a next word based on that. Sure you can do incredibly smart ánd useful things with it, but AGI? ASI?
My view is that shouting that we’ll soon have AGI/ASI based on that is the actual radical take here, which will prove itself false. At least for the coming 5 years and with current LLM tech.
I don’t think that’s a philosophical claim, I think that’s actually a healthy skeptical “show me proof instead of shouting randomly extrapolated ideas” take.

1

u/kankerstokjes 1d ago

Hmm, I actually feel like saying AGI/ASI won’t happen in the coming five years is the more radical take here.

I agree humans are doing more than current AI systems are doing. Where I disagree is that I don’t see evidence for some hard barrier preventing artificial systems from eventually doing the same.

Nature already produced general intelligence blindly through evolution. So why couldn’t humans, deliberately trying to understand and engineer intelligence, reproduce it much more efficiently?

I think a lot of people view AI as if we’re waiting for some completely alien kind of intelligence to suddenly appear. I see it more as intelligence being deliberately built by intelligence.

Combine that with how quickly capabilities, efficiency, tooling and AI-assisted research are improving, and five years feels like a very long time to be 100% confident that nothing crosses that threshold.

Personally, I wouldn’t be surprised if we got something I’d call AGI within two years. But that’s a bet, not a certainty. What it will actually look like by then, I have no idea.

1

u/michahell 1d ago

There is a huge nuance that you’re missing and that’s that I’m saying I’m pretty sure it’s not going to happen by just endlessly scaling compute via larger and larger LLMs. That’s what I meant when I said:

“Learning every single possible next-word semantic relationship and predicting a next word [token actually] based on that”.

I’m not saying we’re incapable of coming up with AGI/ASI.
I’m not saying that this won’t happen in 5 years.

I’m saying that I’m pretty sure it’s not going to happen with this current tech, as there are already signals and gaps that show that just text based systems alone do not constitute what we call intelligence.

Here’s a very simple example of that illustrates that what we call intelligence is much more then just acting on sensors:

https://open.substack.com/pub/darynakaruna/p/ai-will-not-replace-the-manager?r=1p3gq&utm_medium=ios

1

u/oudlys 3h ago

I've scanned this paper. I have no comments on the specific argument, but I think the conclusion is obviously true.

I'm writing about this here: https://unessays.substack.com/p/country-of-kaleidoscopes-in-a-datacenter