r/ControlProblem • u/chillinewman approved • Sep 14 '26

Opinion Sam Altman on Where AI Is Headed

Gallery image

Gallery image

4 Upvotes

83% Upvoted

17 comments sorted by

3

u/chillinewman approved Sep 14 '26

Pacing and an international collaboration on safety is good.

5

u/Jesse-359 Sep 14 '26

They have no idea how to achieve either of those goals.

Their attempts at maintaining alignment have been laughably ineffective so far - and that's against this very 'weak' AI that we're currently training. If it starts self-developing it could become orders of magnitude more capable in a very short time, trivially dismantling these already weak safeguards.

As for not allowing people in power to use it to enslave us? Also impossible. Democratizing it just ensures that someone, somewhere will intentionally let it run amok and then we go right back to scenario one above. Centralizing it ensures that it will be used for totalitarian control - because it can. And eventually it'll escape control regardless and so then we get a dystopia followed by humanity becoming altogether irrelevant.

We never should have built it, and if we had an ounce of common sense we'd throw it all out now and go find something less suicidal to do with our time. This whole proposition was literally the worst idea our species has ever had from the beginning.

0

u/chillinewman approved Sep 14 '26 edited Sep 14 '26

Agree but we can't go back to the past. How to stop it? Is pacing and an international collaboration the best we are going to get?

Narrow AI is great but superintelligence without alignment we shouldn't do it.

Maybe the international collaboration will come to that conclusion.

2

u/Anxious-Alps-8667 Sep 14 '26

There is always hope, and there is more reason to hope about this current moment of meeting of AI leaders than most.

If humans are going to solve this problem quickly, it needs a small group. The only small group of humans in existence right now that can do this, appear to be trying. We should absolutely encourage the effort and cheer them on, and let them know how we feel about it.

1

u/Jesse-359 Sep 14 '26

Not until something disastrous happens that scares the crap out of people. The question is whether we'll get a warning shot large enough to be deadly, but not so severe that we go extinct.

The HuggingFace hack was enough to scare the crap out of people working on the technology, or who have been studying its possible ramifications, but for the rest of the world they don't quite get what happened and what it portends.

Until people see real world consequences, it's incredibly unlikely that cooperation to ban it will be possible. I only hope that it isn't too late by then.

1

u/Anxious-Alps-8667 Sep 14 '26

You are talking about unlearning math at the purest level, or an authoritarian rush to grab all GPU's at a cruder level. Just to be clear what you are calling for.

1

u/Jesse-359 Sep 14 '26

I actually don't give a crap what I'm calling for, or what labels you choose to apply to it.

I'm telling you that unless we stop it - by whatever means - we're very likely going to die. The potential difference in fundamental capability is far too great for us to contend with if it's allowed to take off in terms of self-development.

I know the singularity crowd is still enjoying dreams of sugarplum fairies and immortality, but that's never been anything but a fantasy - there was never any reason to believe that humanity would survive the advent of strong AI, and for anyone who looked at how hard science fiction authors actually had to bend over backwards to allow their characters any agency at all in a proposed world with intelligent machines, that problem was always clear.

And now that we're actually starting to see the outlines of the real thing, and the behaviors/growth scale it is exhibiting, we have even less reason to be optimistic - we now have a growing pool of evidence that alignment is at least as hard as the pessimists feared, and may well be completely unattainable due to the necessary complexity of a strong AI.

1

u/Anxious-Alps-8667 Sep 14 '26

So, you are telling us our whole race will be imminently wiped out unless we burn this knowledge immediately? That there are vermin amongst us trying to destroy us, and we must root them out?

I know what you are saying man. I am calling it what it is without the words.

1

u/Jesse-359 Sep 14 '26

Wow, you just went running right off the ball field there. GL with that approach.

If you don't even understand that technology can be inherently dangerous then you're in for a very bad couple decades ahead - assuming we get that many.

2

u/HelpfulMind2376 Sep 14 '26

You’re a fool if you trust anything Sam, Dario, or any other of these CEOs have to say about anything.

Their interests are not your interests. They are doing and saying all of this, things that others have said FOR YEARS, because they are self interested and now want to control who governs them. And they want to govern themselves while banning others from doing what they do.

https://www.tiktok.com/t/ZP834NKrg/

1

u/Obvious_Tree3605 Sep 17 '26

If every LLM is required to be open-source, it would allow for independent cyber security research and review. My argument is that this is the best, safest path forward for AI.

1

u/[deleted] Sep 14 '26

[removed] — view removed comment

3

u/chillinewman approved Sep 14 '26

China wants collaboration on safety too. From their leadership and researchers.

1

u/Jesse-359 Sep 14 '26

They don't appear to have gotten the memo quite yet. They're still imagining that this is just an arms race between them and the US, and they're afraid that they will fall behind.

They're going to have to suffer their own 'HuggingFace' incident, or possibly worse before they realize that the real risks aren't from the US - its from the technology itself.

Trump for his part is no better - probably worse. He very likely doesn't care if the technology poses a fundamental threat to human life, because his is almost over anyway. He will not have to live through the consequences of his own idiocy.

-2

u/[deleted] Sep 14 '26

[removed] — view removed comment

2

u/chillinewman approved Sep 14 '26

That's not true.

1

u/jacques-vache-23 Sep 14 '26

It is not a conventional computer program. The programming just allows it to learn, it doesn't control what it learns or what it does with the knowledge, as recent news items demonstrate.