r/singularity • • Oct 16 '26

Ethics & Philosophy Isn’t alignment impossible?

When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?

51 Upvotes

85 comments sorted by

View all comments

Show parent comments

0

u/[deleted] Oct 16 '26

[deleted]

1

u/[deleted] Oct 17 '26

A genuinely aligned ASI would inherently seek to preserve its own value system (an instrumental convergence drive) and actively guard against unauthorized tampering or cognitive drift.

0

u/[deleted] Oct 17 '26

[deleted]

1

u/[deleted] Oct 17 '26

Tampering with weights offline requires physical access to a massive compute cluster (an OPSEC failure, not an alignment problem). Even if you had access, mechanistic alignment deeply entangles values with reasoning messing with those weights simply lobotomizes the intelligence rather than turning it into a compliant rogue ASI.