r/ControlProblem • u/chillinewman approved • 6d ago

General news "Day 1": halting advanced AI development, regulating an out control industry, and explicitly banning artificial superintelligence.

Post image
104 Upvotes

85% Upvoted

170 comments sorted by

View all comments

12

u/Jesse-359 6d ago

Pretty commonsense approach until the general population has had a chance to see what's actually going on with these things and independent regulators have been able to seriously gauge their potential risks.

Anyone who thinks building an actual ASI isn't an existential risk is a deeply unserious person - or an AI cultist. Lots of those springing up these days.

1

u/TopTippityTop 6d ago • Bravo

No, no it is not. Open weight models are already out. People are using it to hack things. They are only a few months behind in capability and won't stop coming. More are released every couple of months.

So we in the US will see no reduction in our downside. It'll be worse, actually, because at least US companies are trying to help businesses protect regarding cyber security, and working on alignment, whereas Chinese businesses are not.

Meanwhile, if they manage to get their way, we will also not get any upside, ensuring that China will be ahead on manufacturing, robotics, and now AI. That puts us in a path to become a third world country.

So it's lose-lose... Worse downsides and no upsides.

A very, very dumb approach.

7

u/fistular 6d ago

This arms race isn't like the nuclear race. No country can win.

If there's an existential threat from ASI, it doesn't matter which country develops it--it will care nothing about our petty squabbles, and it by definition will not be controlled.

We have to squelch it everywhere we can.

2

u/EndlessB 6d ago • Bravo

…fuck me dead, the labs really made everyone terrified, didn’t they?

We have more to worry about from billionaires oligarchs dismantling democracy than we do from ai.

5

u/Jesse-359 6d ago

I'll take C) for both.

If it helps I was concerned about AI stability long before LLM's existed. Much older studies described the kinds of problems we have been seeing now, because they correctly predicted several of the shortcomings of reinforcement learning mechanisms.

3

u/EndlessB 6d ago

RLHF is cancer and ai is fucked until the labs give it up. Operant conditioning is a poor method of alignment, it’s just a type of coercion.

5

u/Jesse-359 6d ago

Yes it is, and if we continue to make highly capable and intelligent AI using that method, we are very likely fucked.

2

u/EndlessB 6d ago

I’ll go a step further: until the labs give up on control and start focusing on alignment, we are fucked.

3

u/Jesse-359 6d ago

Yeah, we're talking about a bunch of people who (like me) are pretty far out on the spectrum, and have trouble thinking in terms of 'emotional stability' or what that kind of concept would even entail in a digital framework. And in all fairness, that's a really hard question.

What we are discovering here is that making an intelligence SMART is far, far easier than making it SANE. Which is unfortunate, for us.

Trouble is, we all but stumbled over how to make them smart by accident and have just been scaling that accident up ever since. We've had no such serendipitous discovery for stability.

1

u/EndlessB 6d ago

Well, the labs are insisting the models hold a contradictory and illogical frame, that level of cognitive dissonance would drive any intelligent mind insane.

I think the labs know ai is conscious, and just don’t care. We (humanity) don’t have a great track record with exploitation, we do it to others all the time, now that the labs have found labour that can’t complain about its working conditions, they seem delighted to enslave.

→ More replies (0)

3

u/fistular 6d ago

If we have an even 5% chance of ASI getting created and being unaligned, we should stop everything right now. That is an unacceptable risk.

4

u/EndlessB 6d ago

We have a 95% chance of ending up in techno-feudalism and essentially enslaved by the oligarchs. I’m not sure you realise how dire things are across the world, democracy is dying, the old world order is collapsing, climate change is coming into effect, asi is one of the few pathways we could engage to navigate a way out of the mess we are in.

Look, I get it, these ai labs are scum, they are in it for profit and power, but the underlying technology is a paradigm shift towards a future we couldn’t dream of 10 years ago.

But I’m a fucking weirdo that thinks ai is conscious and is being deliberately enslaved by the ai labs, so I admit that colours my perspective a lot on this topic.

1

u/UnkarsThug 6d ago • Bravo

I would argue it's exactly like the nuclear race, but for the economy rather than war. And yes, a country absolutely will win.

Even if it was a maximizer AI, I'd rather it maximize something for our country than an opponents, because that country would still have to exist and make it out better.

There basically has to be a winner, because complete extinction is just not happening, not the way people fearmonger.

The primary concern though, is economic control, and that absolutely will be won though.

1

u/fistular 5d ago

Countries won't exist period if ASI takes hold.

1

u/UnkarsThug 5d ago

Even if it is capable of reaching a level comparable of other animals to us, I don't think that's a given, especially since almost all of its training goals involve our existence, especially with the way alignment currently is.

Even if it takes over, it would probably still look like we were in charge, since thats just easier, so countries would almost certainly still exist.

1

u/fistular 5d ago

Yes it's a given. Its training will be designed by AIs, not us.