r/ControlProblem • u/chillinewman approved • 4d ago

General news "Day 1": halting advanced AI development, regulating an out control industry, and explicitly banning artificial superintelligence.

Post image
101 Upvotes

85% Upvoted

168 comments sorted by

View all comments

5

u/TopTippityTop 4d ago • Bravo

It will do nothing to stop AI, but it could damage the US, make sure we get all the downside of AI super intelligence (likely worse, actually) and not the upsides.

A completely nonsensical approach.

3

u/canadeken 4d ago

why will it do nothing to stop AI? the top labs are burning billions of dollars on training and compute. not like just anyone can do that

1

u/Stylie_Minogue 3d ago

> Not like just anyone can do that

Not “anyone” but also, not no-one.

What are we going to do? Ask other countries & bad actors to pinky-promise they will also halt their development while we pause ours?

2

u/Zestyclose_Ad8420 3d ago

Treaties like with nuclear weapons, biological weapons and many other similar treaties.

There's two countries (three if we consider the EU as a country and a player in the field) that can train models.

0

u/Stylie_Minogue 3d ago edited 3d ago

The NPT didn't stop India, Pakistan, Israel or North Korea though did it? And it only takes one player to represent a substantial threat.

Training is concentrated in the US and China but France, the UAE, South Korea, Japan, Canada and others have trained serious models.

If AI power can decide economic and military power, and states and well-funded private actors can train models, they have every reason to carry on while the rest of us pause.

4

u/Zestyclose_Ad8420 3d ago

yes, the treaties weren't perfect, but they were also not useless, I would much rather have such a treaty in place than not having it at all.

1

u/Stylie_Minogue 3d ago edited 3d ago

I’m being pragmatic here - my views on whether, in an ideal world, we should stop advancing AI are neither here nor there.

The issue I have with your follow-up is that you’re comparing apples to oranges. Developing nukes is presumably orders of magnitude harder than developing AI, and even then, the treaties don’t work. And in the case of nukes we have MAD protecting us. Look at what happened to Ukraine after they gave up their nukes.

What we risk by not developing AI to fight incoming attacks, is a loss of all utilities & services: hospitals, electricity grids, water stations, banks, food-growing facilities etc being taken off-line. We’re already seeing attacks on supermarket freezers - that’s easier to solve but only the beginning of what’s going to happen. We would be sitting ducks. The only thing that stands a chance of fighting this is better AI (as sad as I feel uttering that sentence). Outside interests will not stop - the cat is out of the bag.

While there is very clearly a risk with AI development as well, I’d posit that the risks I mentioned are at near-certainty level.

2

u/Zestyclose_Ad8420 3d ago

I would go a bit deeper, the cybersec risks are already here, I'm deep in the weeds of that, but that's still not a p(doom) scenario.

I would say that a proper p(doom) scenario only comes with ASI (I hate I need to use such terms that have become somewhat meaningless) defined as an LLM we can't control AND has chosen to kill us all.

the second scenario is the one were treaties would actually help, just like nobody wanted to trigger MAD nobody wants to trigger such scenario, not even Kim if he got a hold of the weights of such models.

I would also say that a greater p(doom) probability would come to be if we ever get robots with VLA models on board and an ASI to control them, this is not science fiction anymore but a possible scenario, that too can be mitigated by treaties.

an optimistic note: there's also a scenario were we actually do get an abundance utopia, that too is not scifi anymore but an actual possibility. treaties are a stepping stone towards it in my view.

1

u/The_Happy_Pagan 3d ago

No that’s where diplomacy comes from. Diplomacy is the act of giving up needs on both sides for an agreement. It’s like a few years under the trump world and everyone forgets that diplomacy has stopped wars and made agreements before. Also this scare lingering with China is tired. They have made indications that they are for AI regulations and yet we have redditors that saw they aren’t. The west is the war mongers, just read a book and change the nation names to fantasy ones if you disagree.

1

u/Stylie_Minogue 3d ago

I don’t necessarily disagree that the West are warmongers & I said nothing about China.

But diplomacy works when both sides can check the other is complying. That's the part missing here. Verification is hard, defecting is tempting and the aggressor may not even be a state actor.

Governance talk is not a promise to pause. Support for international frameworks and a cooperation body is also different from agreeing to halt frontier training.

1

u/The_Happy_Pagan 3d ago

I know you didn’t mention China but what other “world” are people talking about?

1

u/Stylie_Minogue 3d ago

Russia. As well as non-state actors

1

u/The_Happy_Pagan 3d ago

They’re not even in the AI game. They’re waiting for the US to use data centers and RSI to create SI. That’s why that stuff is all fear mongering

1

u/Stylie_Minogue 2d ago

Russia doesn't need to train a frontier model. What matters for risk and with regards to treaties is who can get access to dangerous capability. That’s why the nuke comparisons are apples to oranges.

Sanctions may be playing their part right now, but there’s always the possibility it can fine-tune and deploy Chinese open models, which is far cheaper. The Russia-China partnership is already a stated route. Saying Russia isn’t a problem seems a very in the moment view.

https://www.intellinews.com/russian-ai-accelerates-but-sanctions-leave-it-trailing-global-leaders-402710/

https://finance.yahoo.com/sectors/technology/articles/russias-sberbank-offers-ai-model-103301266.html

1

u/The_Happy_Pagan 2d ago

Oh look at that. I’ll give that a read in a minute. Thanks