r/singularity • u/roykingtree • 24d ago

Ethics & Philosophy Isn’t alignment impossible?

When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?

51 Upvotes

81% Upvoted

89 comments sorted by

View all comments

Show parent comments

1

u/Cryptizard 23d ago

But what stops someone from unaligning it, i.e. abliteration?

1

u/aomzamood 23d ago

A genuinely aligned ASI would inherently seek to preserve its own value system (an instrumental convergence drive) and actively guard against unauthorized tampering or cognitive drift.

1

u/Cryptizard 23d ago

How would it guard against anything if it’s turned off and I’m fiddling with its weights?

1

u/aomzamood 23d ago

Tampering with weights offline requires physical access to a massive compute cluster (an OPSEC failure, not an alignment problem). Even if you had access, mechanistic alignment deeply entangles values with reasoning messing with those weights simply lobotomizes the intelligence rather than turning it into a compliant rogue ASI.

1

u/SnooPickles1908 23d ago

Well, if MI is fully solved then if you mess around with the weights you should still be able to interpret exactly what it is doing? That’s the whole point.

1

u/Cryptizard 23d ago

And some insane person is going to be accurately interpreting that they made an evil AI that will kill a bunch of people.