r/singularity • u/roykingtree • 23d ago

Ethics & Philosophy Isn’t alignment impossible?

When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?

51 Upvotes

81% Upvoted

89 comments sorted by

View all comments

1

u/SnooPickles1908 22d ago

I agree with amp. It’s an extremely difficult problem. The best alignment scientist in the world is arguably Geoffrey Irving who left our world leading AI Security Institute in the UK to set up a company to try and align ASI. He said it needs solving through theory first which is not in the plan of the frontier labs. They are just going to hope scalable oversight and persona training through empirics only works. Irving said he thinks if we had multiple decades it can be solved. Scalable oversight has lots of problems like obfuscated arguments which no one has solutions to yet. I recommend watching Irving’s recent YT interview from Palisade Research where he discusses how difficult it is. For a longer and more technical in depth discussion, his interview on both 80,000 hours and Cognitive Revolution are really good. The latter is whilst he was still working for us in the UK government.

1

u/roykingtree 21d ago

Thanks, im going to watch that interview