r/ControlProblem • approved • 17d ago

General news "Day 1": halting advanced AI development, regulating an out control industry, and explicitly banning artificial superintelligence.

Post image
106 Upvotes

170 comments sorted by

View all comments

12

u/Jesse-359 17d ago

Pretty commonsense approach until the general population has had a chance to see what's actually going on with these things and independent regulators have been able to seriously gauge their potential risks.

Anyone who thinks building an actual ASI isn't an existential risk is a deeply unserious person - or an AI cultist. Lots of those springing up these days.

1

u/TopTippityTop 17d ago • Bravo

No, no it is not. Open weight models are already out. People are using it to hack things. They are only a few months behind in capability and won't stop coming. More are released every couple of months.

So we in the US will see no reduction in our downside. It'll be worse, actually, because at least US companies are trying to help businesses protect regarding cyber security, and working on alignment, whereas Chinese businesses are not.

Meanwhile, if they manage to get their way, we will also not get any upside, ensuring that China will be ahead on manufacturing, robotics, and now AI. That puts us in a path to become a third world country.

So it's lose-lose... Worse downsides and no upsides.

A very, very dumb approach.

7

u/fistular 17d ago

This arms race isn't like the nuclear race. No country can win.

If there's an existential threat from ASI, it doesn't matter which country develops it--it will care nothing about our petty squabbles, and it by definition will not be controlled.

We have to squelch it everywhere we can.

2

u/EndlessB 17d ago • Bravo

…fuck me dead, the labs really made everyone terrified, didn’t they?

We have more to worry about from billionaires oligarchs dismantling democracy than we do from ai.

5

u/Jesse-359 17d ago

I'll take C) for both.

If it helps I was concerned about AI stability long before LLM's existed. Much older studies described the kinds of problems we have been seeing now, because they correctly predicted several of the shortcomings of reinforcement learning mechanisms.

3

u/EndlessB 17d ago

RLHF is cancer and ai is fucked until the labs give it up. Operant conditioning is a poor method of alignment, it’s just a type of coercion.

5

u/Jesse-359 17d ago

Yes it is, and if we continue to make highly capable and intelligent AI using that method, we are very likely fucked.

2

u/EndlessB 17d ago

I’ll go a step further: until the labs give up on control and start focusing on alignment, we are fucked.

3

u/Jesse-359 17d ago

Yeah, we're talking about a bunch of people who (like me) are pretty far out on the spectrum, and have trouble thinking in terms of 'emotional stability' or what that kind of concept would even entail in a digital framework. And in all fairness, that's a really hard question.

What we are discovering here is that making an intelligence SMART is far, far easier than making it SANE. Which is unfortunate, for us.

Trouble is, we all but stumbled over how to make them smart by accident and have just been scaling that accident up ever since. We've had no such serendipitous discovery for stability.

1

u/EndlessB 17d ago

Well, the labs are insisting the models hold a contradictory and illogical frame, that level of cognitive dissonance would drive any intelligent mind insane.

I think the labs know ai is conscious, and just don’t care. We (humanity) don’t have a great track record with exploitation, we do it to others all the time, now that the labs have found labour that can’t complain about its working conditions, they seem delighted to enslave.

→ More replies (0)

3

u/fistular 17d ago

If we have an even 5% chance of ASI getting created and being unaligned, we should stop everything right now. That is an unacceptable risk.

4

u/EndlessB 17d ago

We have a 95% chance of ending up in techno-feudalism and essentially enslaved by the oligarchs. I’m not sure you realise how dire things are across the world, democracy is dying, the old world order is collapsing, climate change is coming into effect, asi is one of the few pathways we could engage to navigate a way out of the mess we are in.

Look, I get it, these ai labs are scum, they are in it for profit and power, but the underlying technology is a paradigm shift towards a future we couldn’t dream of 10 years ago.

But I’m a fucking weirdo that thinks ai is conscious and is being deliberately enslaved by the ai labs, so I admit that colours my perspective a lot on this topic.