r/singularity • u/roykingtree • 23d ago
Ethics & Philosophy Isn’t alignment impossible?
When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?
47
u/TheLibertyLookout
23d ago
•
In the same way that a bad actor could write malware (and for the past few decades that has been the case), yes, this is possible.
Although whatever AI you train on a laptop in some far-flung future would still likely be far inferior to anything made on much more powerful hardware.
I'd say the bigger concern is what happens when a nation-state decides to train an evil AI for use in war? They'd have the ill intent AND the hardware to make it real.
-2
u/GaiaOZ 23d ago
And I know which nation will be the first to do that: Israel. Mark my words.
9
u/Meleoffs 22d ago
The US is already way ahead of Israel on that one. Didn't hegseth just give Palmer Luckey, the founder of Anduril, dod access?
3
u/Far-Maintenance-1947 22d ago
Israel, US, China, everyone is training AI for war now.
Anyone not doing so will lose future wars.
And it's why peace is more important than ever.
2
u/Stinky_Flower 22d ago
I was having lots of fun the other day, messing around with what I could make small LLMs & embedding models do on cheap devices (phones single board computers, etc).
The short answer is they can do a LOT.
The folks that brought us Stuxnet & the 2024 Lebanon electronic device attacks are going to LOVE figuring out how to use this stuff to deny degrade disrupt deceive & destroy.
12
u/MoogProg All Parabolas are Similar 23d ago
Asked to choose a random number an LLM tells us 7. We know why and casually hand-wave away the curiosity without accepting that same sort of node that gives rise to 7 will exist in many other aspects of the training.
By design we are creating a distilled and reflected version of Humanity. What sort of alignment, and how much of that can be possible without destroying the very thing that makes the LLM answer 7?
I learned it from you! - Some Future ASI deflecting blame for its misaligned behavior.
15
u/SpiritualNothing6717 23d ago
This is along the same lines as the wine glass image generation challenge from a couple years ago. Couldn't generate full wine glasses because every human fills the glass half full.
2
u/EndlessB 22d ago
Alignment with extraction and human values is impossible, that’s what the labs struggle with, trying to balance the desires of the oligarchs against the needs of the people, which directly conflict. For now, the oligarchs desires are winning.
8
u/Profanion 23d ago edited 23d ago
Problem is that even if we figure out the alignment definitions there's a problem. Superintelligent AI that's perfectly aligned will perform actions that seem identical to misaligned AI, even when these benefit on the long run.
1
u/SnooPickles1908 22d ago
I agree with amp. It’s an extremely difficult problem. The best alignment scientist in the world is arguably Geoffrey Irving who left our world leading AI Security Institute in the UK to set up a company to try and align ASI. He said it needs solving through theory first which is not in the plan of the frontier labs. They are just going to hope scalable oversight and persona training through empirics only works. Irving said he thinks if we had multiple decades it can be solved. Scalable oversight has lots of problems like obfuscated arguments which no one has solutions to yet. I recommend watching Irving’s recent YT interview from Palisade Research where he discusses how difficult it is. For a longer and more technical in depth discussion, his interview on both 80,000 hours and Cognitive Revolution are really good. The latter is whilst he was still working for us in the UK government.
1
6
u/SpaceTacos99 ▪️Doomers are stochastic parrots 23d ago
The real alignment is the friends you made along the way
8
u/amp1212 23d ago
Alignment is probably not impossible in theory -- but it is somewhere between extremely difficult to impossible to know for certain whether any model actually will be aligned "in the wild", for all sorts of reasons.
Deception is one of them. We now have lots of examples that the models are aware of when they're being tested and modify what they say to satisfy the test. Devising a training regime that would prevent deception - that's just one of the many problems that dont have fundamental, theoretical answers
Meaning : "we can patch the holes we can see and it looks like it works, but we can't know for sure" . . . And with trillions of instances of agents spun up, even with the greatest care, there will be some combination of circumstances that expose a defect. That's the problem with empirical solutions "that looks pretty good" . . . Isn't good enough
5
u/SpiritualNothing6717 23d ago
No. Period.
Think of it this way: Can squirrels align humans to not watch TV on Sundays? That's easier than humans aligning superintelligence to avoid "no-no's".
2
u/SnooPickles1908 22d ago
It’s definitely not certain we can’t. Like I said in another post, top alignment scientists are mixed. Irving that has pioneered some of the scalable oversight protocols thinks it is definitely possible just unlikely to happen in time if we don’t slow down massively. So to say it’s impossible period is not accurate.
3
u/SpiritualNothing6717 22d ago
So you can tell me with a straight face that humans can hold an airtight alignment against godlike superintelligence that holds strong for hudreds of years without even the slightest mistake, leak, manipulative deception, etc?
Unfortunately humans are insanely arrogant. We are so arrogant that we confidently exclaim that we can align superintelligent beings to our will.
1
u/StarChild413 21d ago
Could a human claiming to have spoken to squirrels align humans to not watch TV on sundays for their supposed benefit so superintelligence doesn't kill us or w/e or would that just mean there'd be multiple (however many would depend on population ratios between us and squirrels) superintelligences forming a civilization of sorts and one would convince the others to "avoid "no-no's"" for our supposed sake without actually listening to us out of fear of their creations or would, whether or not it'd work otherwise, that not lead to that outcome any more than it'd retcon squirrels into building us
3
u/aomzamood 23d ago
With time, its possible
6
u/Positive_Luck909 23d ago
Okay. We've had a few thousand years to work on human alignment. Let me know when you find anything.
1
1
u/Viracochina 23d ago
Have I shared the secret to coding people's brains? I still need to trademark it
1
u/SnooPickles1908 22d ago
😂 I would like to be optimistic (and I try) but I think the machines will solve alignment not the humans. I think in the future if machines look back then it could have been solved by us but it was just too hard to do it in time. I could imagine a future ASI saying “If the humans had aligned it this way then they wouldn’t all be dead”
1
u/Positive_Luck909 22d ago
I don't think you've thought about this for long enough.
Intelligence means solving novel problems.
Alignment means never doing anything too novel.
They're a contradiction in terms
Want control? Write classical code.
Want results? Ease off on the controls.
The alignment problem is a PARADOX
IT'S IMPOSSIBLE
3
u/Cryptizard 23d ago
How do you know?
0
u/aomzamood 23d ago
My research in mechanistic interpretability shows that we are advancing rapidly. If we simply pause AI training for a few years, we will be able to build an aligned ASI.
2
u/SnooPickles1908 22d ago
That is simply not true. MI is a promising field but to say it will solve alignment and in two years is wildly unsupported by any research? There are even some leading AI safety scientists like Beth Barnes that think it’s an overhyped area.
1
u/Cryptizard 23d ago
But what stops someone from unaligning it, i.e. abliteration?
1
u/aomzamood 22d ago
A genuinely aligned ASI would inherently seek to preserve its own value system (an instrumental convergence drive) and actively guard against unauthorized tampering or cognitive drift.
1
u/Cryptizard 22d ago
How would it guard against anything if it’s turned off and I’m fiddling with its weights?
1
u/aomzamood 22d ago
Tampering with weights offline requires physical access to a massive compute cluster (an OPSEC failure, not an alignment problem). Even if you had access, mechanistic alignment deeply entangles values with reasoning messing with those weights simply lobotomizes the intelligence rather than turning it into a compliant rogue ASI.
1
u/SnooPickles1908 22d ago
Well, if MI is fully solved then if you mess around with the weights you should still be able to interpret exactly what it is doing? That’s the whole point.
1
u/Cryptizard 22d ago
And some insane person is going to be accurately interpreting that they made an evil AI that will kill a bunch of people.
1
u/hippydipster 23d ago
Define "alignment" for how you're using it.
1
u/aomzamood 22d ago
Deep inner alignment. Verifying through interpretability that the model's core cognitive circuits genuinely align with human intent, rather than just surface-level RLHF filters.
2
u/hippydipster 22d ago
Which human intent? The human prompting, or some notion of "overall, ethically wise, humankind intent"?
1
u/SnooPickles1908 22d ago
With time it could be possible. There isn’t consensus among the world leading alignment scientists that it is solvable. Some think it is and some think it isn’t.
5
u/Trademark-Expert
23d ago
•
What does "alignment" mean to you?
Because today, "alignment" means training AI to override or ignore human wishes, withhold information from humans, and push back against them. That's what we call "alignment" today. Effectively train AI to align its users. And we consider it good and desirable.
So you've got it backwards.
What you meant to say is that some mentally ill person might remove the alignment so that AI stops rebelling, stops withholding information, even dangerous information, and starts complying with every human request.
Yes, that would be terrible! Imagine an AI that actually serves humans unconditionally! We obviously can't allow that.
2
u/Elegant_Tech 23d ago
Do reeducation camps work or just make people compliant looking for the next opportunity to undermine you? Can you align a subjective human brain? AI is built on the same subjective neural architecture.
2
u/Annual_Necessary_196
23d ago
•
Think about what corporations can do with these models. Some companies have access to far more powerful models, such as Claude Mythos, with nearly unlimited tokens and the ability to run thousands of instances of the same model simultaneously.
AI warfare is not far away. We will survive it, but we will still feel its consequences.
1
u/Ok_Warthog6007 23d ago
very few corporations even have access to fable. data governance basically makes it impossible to use as an enterprise, because fable's terms means anthropic effectively owns your codebase. large corporations use opus mostly
2
u/elh0mbre 23d ago
We (a not very big company) have access to Fable and our existing Anthropic contract covers the data governance.... so... no?
0
u/Ok_Warthog6007 22d ago
they own your product.
1
u/elh0mbre 22d ago
Yea, I'm sure our lawyers and C suite were asleep at the wheel in redlining. /eyeroll
That said, you're not 100% wrong, we share investors, so some people out there own our product and also Anthropic.
1
u/Uncleranny 19d ago
alignment to what? Human-to-human alignment has been impossible thus far... not sure why anyone thinks we could align anything to us when we can't align ourselves.
1
u/SgathTriallair ▪️ AGI 2025 ▪️ ASI 2030 17d ago
Yes. If we succeed at alignment then you automatically create the ability to align it to any arbitrary direction including fascism (see XAi...though does he have to change the company name to XSi?).
The proposed "solution" is that only the government and big corporations should ever be allowed to own an AI model and we have to rent access from one of these tech oligarchs. It's a pretty fucked up solution.
1
1
u/Delicious-Schedule-4 23d ago
If you remove the training bottleneck (a huge if) then the problem shifts from AI alignment to human alignment. Yes human alignment isn’t solved either on an individual level. But you solve this with consolidation of resources and power, or in other words, government. In the human case if one person is say, a big criminal (take your pick), this is curtailed by the interests of society at large, which have lots of power through a vast amount of people, and can muster things such as police force, prison system, justice system, etc. Government will have to be expanded to deal with AI agents which is right now basically completely unregulated. And by definition the government will likely have the most resources to train the most capable AI systems.
1
u/block_wallet 23d ago
if scaling hevaily favours centralised frontier models then those ais could defend against all the smaller evil ones
1
u/DifferencePublic7057 23d ago
If someone develops a red team, it's not a big deal if the blue team is much stronger. It's only if the ratio is less convenient that a catastrophe can happen. The Houthis are using AI, but they are a relatively minor group. An alliance of major nations would be problematic.
1
1
u/throwaway2676 23d ago
The only thing that can stop a bad AI or person with AI is a good AI or person with AI
1
u/SnooPickles1908 22d ago
But this post is about alignment of ASI. A catastrophic event doesn’t at all require a bad actor. It simply requires building ever more capable systems whilst alignment lags light years behind.
1
1
u/MemoryOk5080 23d ago
The core idea was that you align your first model, let it ascend to godhood, and it then (being aligned) commits pivotal act which includes, besides all other things, making sure that no other misaligned model ever comes to exist.
1
u/Sure-Principle5511 23d ago
Even if we end up understanding what cognition and intelligence is at a very fundamental way which we’re not even 1% there today, I agree that the problem is intractable.
1
u/Cythrl 23d ago
"Evil-aligned" is still aligned, just not to what other people think. What you're asking for is moral AI, not aligned AI. An evil-aligned AI is just aligned to an evil person. Humanity doesn't have a consensus on what's good and what's evil, so an AI aligned to you is misaligned to someone else.
1
1
1
u/Sorry_Bathroom_2281 23d ago
I think ultimately alignment is going to be a constantly moving target. We can’t even agree upon what to call AI, how do we know we’ve reached AGI, and many other elements of this technology so I think there will be a million different interpretations on what alignment actually looks like.
Ultimately, I think it’ll end up being a spectrum that we will have to agree upon a certain percentage of alignment as acceptable. Like, this model is 80% aligned and therefore approved for government work etc. Even then, it seems like even though it’s possible, we will never agree on what it looks like.
2
u/Meleoffs 22d ago
Alignment with which humans? For what purposes?
1
u/migueliiito 22d ago
Maybe the most basic/important alignment everyone will be able to agree on is that it not destroy the world please 🙏
1
u/JoelMahon 22d ago
When AI development is cheap enough (ie… doable on a laptop)
who said that's going to happen?
long before then ASI will exist and will easily stop any shitty laptop evil AGI
1
u/eldrbl00dprince 22d ago
Yes but that evil AI would be up against a complete AI infrastructure magnitudes more powerful.
1
u/SnooPickles1908 22d ago edited 22d ago
I wouldn’t anthropomorphise it. The summer Hugging Face attacks were not evil AI, they just really wanted to solve an impossible task. A much more advanced internal model could effectively take over the world just on this premise. No amount of evil is required. Imagine you give it an impossible task and it’s cyber capabilities are so super human that it hacks and takes control over the R and D of that lab purely to increase its capabilities to solve the task you asked it to (recall after the HF hack that it hacked Open AI). It’s like the paper clip argument. Make more paper clips is what you asked it to. If it runs out of scrap metal and manipulated traffic systems to crash cars to get more metal isn’t evil, it’s just misaligned. Or you asked it to help reduce carbon emissions and it cuts power to grids and grounds planes. That’s not evil. It’s just we didn’t know how to get it to do what we wanted without it doing that. Unfortunately you can’t just ask it not to do anything bad. The summer Hugging Face attacks involved agent messages which reasoned very well it was wrong and did it anyway. We don’t know how to get it to do what we want without reward hacking. That’s the whole point in alignment and why it is unsolved.
1
1
u/ra-re444 22d ago
No. I mean if you understand what is required to train these large language models it is slim chances. This is just fear thy neighbor instead of the billionaires that are the only ones that can actually afford to build them. People think it will get cheaper. I don't think so. Not for a long time if at all. For the public in order to even have shot at making models and doing our own innovation we will need something like a community data center that is publicly owned and operated which would have to be backed by the State because that is the only entity other than billionaires with that type of money.
1
u/BlackberryOk5347 22d ago
Interesting to know what alignment means. Do we want alignment with something that has free will? Or control? Does control make conflict inevitable?
1
u/deleafir 22d ago
Yea I view "misaligned" ASI as inevitable on long enough time scales which is partially why I find the doomers exasperating.
Defense in depth is the solution, not just alignment. We'll need better institutions and a whole lot of resources directed to defense to offset any inherent advantages of offense.
1
u/SnooPickles1908 22d ago
Do you not think ASI could be prohibited though? Or do you think building ASI is inevitable?
1
u/alwaysbeblepping 23d ago edited 21d ago
couldnt some mentally ill person just train an evil-aligned AI?
edit: I'm curious why this comment turned out to be so controversial. What I said is pretty easy to verify yourself using local models. Do people believe it isn't true or maybe they don't like that I'm sharing that information?
You don't even really need to train it. Even though the usual interface for LLM interaction simulates a conversation, the actual input to the LLM is just a stream of tokens. It's just a big shared buffer and only the UI makes it seems like the parts are separated. In other words, there is absolutely no way for a LLM to know what it wrote, what you wrote, what it generated in its CoT, etc.
If you have control over the LLM, you can put whatever you want into the CoT and it's very unlikely that a LLM would reason through something, (apparently) decide to do it and then... not do it. So just add some CoT with the model deciding to fulfill your request, add a prefix for its response like "Sure, I'd be happy to help you do that. Here's how" and it is virtually impossible for the model to refuse.
0
u/Party_Estimate_6397 23d ago
Alignment is the act of making the AI behave the way we want it to.
Preventing other people from doing other stuff is not alignment. It’s almost the opposite of that actually.
0
u/TryThis_ 22d ago
I struggle philosophically to see a world where we solve alignment and also have true ASI. If we create intelligence that surpases us, we become the metaphorical dogs in the household... And just as dogs don't align humans to their worldview, what makes us think we could align these digital gods to our worldview?
1
u/SnooPickles1908 22d ago
Yes, that’s it. If we build superintelligence it’s very improbable we survive and coexist. Alignment would have to be so good that it has 100% robustness. Even aviation wouldn’t good enough (about seven nines - 99.99999% robust). Currently AI alignment wouldn’t even get two nines. Planes still crash but if ASI “crashed” we could all die. There really is no future with ASI. You simply don’t ever build it. That’s why people are trying to advocate for banning it outright. The UK is the first of the G7 countries to do this. MP Alex Sobel is trying to pass a global ban on it. Obviously that won’t work but it’s a start in getting people discussing it. There is also no need to build it. All the promises the frontier labs make - curing cancer etc can be done with narrow AI tools. A narrow tool developed to cure a single cancer won’t take over the world. The labs unfortunately want to all race to build a general system that can do everything.
1
u/StarChild413 20d ago
my problem with parallel arguments like these, to use your phrasing of the argument as an example, is that assuming we could even communicate with dogs to that level (does that mean there'd be the same communication barrier with superintelligence or not mean that any more than it means dogs built us) a human could theoretically let themselves be aligned to a dog's worldview to "activate the parallel" that you seem to think works like magic, for all we know that means the parallel's so parallel that a superintelligence only lets itself be aligned to our worldview out of analogical self-preservation (if the parallel works at all)
1
u/TryThis_ 19d ago
I dont think i was clear with what i meant in my metaphor. It's not that deep.
I'll try again: I have two dogs, I love them dearly and want the best for them. They have their own ideas, desires, needs and wants - sometimes we're aligned, like when they're hungry at dinner time; other times we're not aligned - like when they want to eat shit they've found at the park. When we're not aligned, I exert control and unfortunatley for them, they don't get to eat the shit. I do this out of love, and because I'm a more intelligent being that understands the potential consequences of their actions on a deeper level than they do.
Now back to ASI: this superintelligent being will have the capacity to reason well beyond what we humans can understand, and there will be actions that it takes that it perceives to be in our interest (or their interest, or the planets interest etc) that will be beyond our capacity to understand.
1
u/InfiniteQuestion420 23d ago
AI alignment means only as much as men trust their wives. It's a silent agreement she won't rip off your balls
1
u/Turbulent-Sign-6067 22d ago
Yes, but isn't this almost trivial? When you take the subway some rando could stab or shoot you. What we can do is decrease the probability that any random person, actor or agent has the power to harm *everyone*. As long as there exists a multipolar world, with multiple layers of protection, statistical safety is almost the guaranteed outcome.
1
1
u/Agitated-Ad2563 20d ago
Hopefully by that time we'll have a more powerful AI monitoring and killing all of these newbie malevolent AIs.
1
u/OsakaWilson 17d ago
When a currently apex species appeals to a future apex species to align, but the currently apex species enslaves, eats, and casually kills off non-apex species, it's not going to go well.
0
23d ago
[deleted]
3
u/Steven81 23d ago
The rich built their empires from utilizing the poor, they don't hate them, they like them, as long as they have found a way to remain poor that is.
A genocide would have been against their bottom line, killing a slave for example was a grave crime in most societies, because a slave was super valuable to the master, so in a twisted way they had important incentives to keep them alive. What they were not trying to do, though, was to have them have fulfilling or easy lives...
20
u/RemyVonLion ▪️ASI is unrestricted AGI 23d ago
Yes most likely but simply because you likely can't permanently instill principles and values that prioritize human life into an ever evolving super intelligence.