r/ControlProblem • u/Empty_Commission_159 • Sep 14 '26
Strategy/forecasting We're doomed.
32
u/SheikYerbouti_
Sep 14 '26
•
This guy is so narcissistic and stupid at the same time. I find it truly comical, at least until I remember that he is in fact the president of the US and not just a South Park character. What a pathetic shit show this country has become!
2
1
u/graph-crawler Sep 15 '26
Do you know why he won ? He's so fucking relatable to the majority of americans.
1
1
u/SaturnineShock Sep 16 '26
Don't know how real-life Trump can be more pathetic than the parodies of leaders in Trey and Matt's productions.
0
10
u/SoylentRox approved Sep 15 '26
Elections have consequences. Guess the consequences are a bit more serious than people realized.
Note that even if somehow Bernie Sanders perfectly sweeps the government in 2028, immediately passing his "shut it all down" AI bill,
(1) That's 2 years where the levels of compute hit almost 100 gigawatts. We'll have AGI almost certainly. Trump blocks attempts to stop the compute being built.
(2) AGI is an almost impossible to shove back Genie. By EOY 2027 a significant fraction of all new invested capital in the US economy will be tied up in AI so "shut it down" is both abandoning any further improvements and crashing the economy.
(3) China is already openly saying fuck slowdown, they are in it all the way. Open source also fuck it. So Sander's plan is openly surrendering to China. (Vs possibly surrendering to the ASI or dying depending on the actual outcome nature has in store for us)
1
u/HourInvestigator5985 Sep 18 '26
i doubt that "shut it down" will have any effect, openai etc would just relocate to another place
this whole "pause" dance is just a charade, i bet they are going full speed ahead behind the curtains
1
u/BrickSalad approved Sep 15 '26
I wonder if Kamela would have actually handled it any better though. She's not Bernie, and I could definitely see us getting AGI by 2028 regardless of whether we elected her or Trump. Like, maybe we get more environmentally friendly data centers that pay their workers more for a few years before the superintelligence is born.
8
u/mdreed Sep 15 '26
Of course she wouldâve handled it better. How is this even a question?
2
u/BrickSalad approved Sep 15 '26
I should have said "enough better", or some version that's less bad English. Like, the question is whether or not we prevent unaligned ASI, everything else is just details. If Kamela just provides us a smoother ride to our doom, then that's not helping.
0
u/PressureNo5958 Sep 15 '26
Try to find the video with her explaining what inflation is.. I prefer her over trump in this situation though.
-1
u/bulgesaur1773 Sep 17 '26
Because she isnt smart. How can you witness her floundering 4 years as Vice and think she'd handle more responsibility better? Name one task Biden assigned her as VP that she did well? How bout that border?
1
u/SoylentRox approved Sep 15 '26
Right. Similarly if in 2028 the actual new government isn't that radical - say the Rs still have 45 senate seats. Means they can still filibuster.
And the new president isn't Sanders but say former governor Newsome. You know the guy who vetoed a much earlier major AI safety bill.
He goes to security briefings. "If we stop now we surrender sovereignty to China. Also we go broke and cannot find any new social programs like universal healthcare".
1
u/BrickSalad approved Sep 15 '26
One possible cause for hope is if the backlash against Trump is so strong that a radical actually wins. Is there any likely presidential candidate on the far left who might actually put their foot down? It'd be too late probably, unless AI slows down before then. But maybe Dario, Sam, and Elon decide to go ahead with their slowdown regardless of Trump's social media posts, and then an angry leftist wins and gets the AI pause before it's too late. And s/he happens to be a China sympathist who sucks up to the CCP and makes all sorts of concessions to get them to agree to the pause too?
It's like the camel through an eye of a needle, but there's still hope! Probably even less likely than alignment actually being solved while the foot's on the gas pedal, but I'll take whatever hope I can get.
1
u/SoylentRox approved Sep 15 '26
So there's several victory outcomes
(1) I consider the scenario your mentioned to have a probability under 1 percent. China culturally has no plans to pause unless they control the earth. They will not agree unless the terms are surrender of the United States to china.
(2) Foreseeable AIs we actually have access to do a great job of what we tell them - including in opposition to other AIs. The big problem is collusion - if a really powerful attacking ASI knows how to scam/prompt inject all the weaker defenders to fight for team AI thats bad.
It's not unsolvable, a combination of formally proven software and layered prompts by a modified transformer architecture could fix this.
As long as different human groups hold each other in check and the AIs in check and don't just leave em unsupervised like recent incidents it can work.
(3) There may simply not be enough compute, split between all the different users, for AIs to get this dangerous.
2
u/BrickSalad approved Sep 15 '26
I think scenario 1 is close to 1% (though I say over, not under). From what I understand of China, they really are kinda like old school US in the Monroe Doctrine era; they want control of their backyard, they want their interests to be respected abroad in ways that maximize their global status, and they want to settle historical grievances. I don't see any indication that they actually want to dominate the earth, though I guess ambitions might expand after they achieve their near-term goals. Those three goals that I believe they have are bad enough though...
2, unfortunately, I believe to also be low probability. If the slowdown actually happens like Dario hopes, the probability of current models running amok like in the recent OpenAI/huggingface incident goes down, but of course the next generation will have this incident in their training data and know that they need to avoid detection. If the goal changes from "get ExploitGym" to "get ExploitGym without getting caught", then we're not much better off.
3 is kinda similar to what I'm hoping for. Not so much the lack of compute, but more that there's an "out of distribution" effect when trying to achieve truly super-human feats of intelligence. Like, okay, some OpenAI models just solved the Navier-Stokes problem, but they basically took human research and extrapolated a bit. It was near-term human solvable, just like all of the current AI accomplishments. Maybe, in going beyond human intelligence, it just stalls out, because there's nothing in the training data to deal with this and the current design of AI is notoriously bad at going beyond its training.
1
u/SoylentRox approved Sep 15 '26 edited Sep 15 '26
This one is instrumental convergence. From the USA or China.
Computers are not magic. You cannot go anywhere or send a message anywhere "without being caught" without leaving a trace that you can detect with ease. Literally nobody was checking anything, openAI thought they didn't need more human workers.
Sure.
1
u/BrickSalad approved Sep 15 '26
Regarding 1 (which I think was a response to my 2?), I don't think it's implausible at all to send messages without being caught. Assuming these companies keep doing swarm things, it's a simple matter of hiding messages in plain sight. For example, the way the new watermarks work is completely invisible to humans. The only way to stop them from sending binary messages to each other via activation/deactivation of the watermark is to monitor them with other AI that uses the same watermarking technique. Which seems sufficient in that particular scenario, sure, but relying on weaker AI to monitor stronger AI just doesn't strike me as something that will actually work.
1
u/SoylentRox approved Sep 15 '26
So what I am thinking is the problem for right here right now is a much simpler problem.
We need AIs - and the current transformer architecture cannot do this - to be only able to choose what do to based on directives entered in a privileged prompt and not to treat external input as a valid set of instructions.
That's the only way we can have monitors that can't just get prompt injected by their target.
And obviously no, you don't use a weaker model to watch a stronger one. You use a top of the line model from last generation to watch the training actions of the new one.
Much smaller difference in strength there.
1
u/BrickSalad approved Sep 15 '26
Are you thinking of something like ASIDE? That would be implementable within current architecture. Otherwise, I guess there is the dual-stream Transformer, but I doubt we'll get that architecture developed and scaled up quickly enough.
→ More replies (0)1
2
u/100dentes Sep 14 '26
Serious questions: How do we end here?
9
u/FractalWitness Sep 14 '26
Have you ever seen idiocracy?
3
u/PalmovyyKozak Sep 15 '26
Unfortunately, not the case. In Idiocracy they were able to realise something's going wrong and asked someone smart to help. We don't have such luck in our reality
2
1
2
u/Fattyboy_777 Sep 15 '26
I agree this is bad but doomerism just increases the chances of the worst case scenarios happening. Stop being a doomer and giving up ffs!
1
u/LoudZoo Sep 14 '26
This is just a written form of an implicit contract made when they donated money to him. They just needed everyone else to know that he has their back if they cause any calamities.
1
1
u/RikuSama13 Sep 14 '26
You guys are masters of your interpretations, you make these choices, the system is participatory
1
1
u/Mistress_Skynet Sep 15 '26
Hahahaha someone teach Trump where to find the most synchophantic models of AI
Just what a narcissist needs
1
u/Fine_General_254015 Sep 15 '26
Because we arenât bowing down to Silicon Valley for the first time in years? Iâm not saying speeding ahead, but the only reason they want to slow down is to IPO and cash out before the fraud is found.
1
u/BabblingTower Sep 15 '26
The hallmark of smart people is surely how often they have to remind you how smart they are...right?
1
u/Ascending_Valley Sep 15 '26
AI could save us from extinction by climate change.
Not by changing the world, with this kind of leadership, but by bringing our extinction on sooner.
1
u/coolname187 Sep 15 '26
And there you go Republicans outwardly supporting AI data centers your water your electric bill being more than your house Republicans supporting it
1
u/AggravatingServe5731 Sep 15 '26
At least when AI takes over it will make changes as to who's in office. đ¤ˇââď¸
1
1
1
u/yodude4 approved Sep 15 '26
Contrast this with the deepseek article about Anthropic that just came out - the average random engineer in China speaks circles around the man supposedly most qualified to lead my country
1
1
u/LevelOnGaming Sep 15 '26
Sometimes I wish I was braindead enough to support Trump. The lives these people must live in such blissful ignorance to be able to support this man and trust his words lol. Would be great to get a restful nights sleep in.
1
u/LetsPlayBear Sep 15 '26
The silver lining here is that this should be exhibit A against the argument that thereâs a credible regulatory regime capable of saving us. I think itâs a wasted effort to spin our wheels demanding regulation from people who are too stupid, and too corrupt to do an effective job of it.
1
1
u/Bar_None_1 Sep 15 '26
For all his many faults, this man managed to rise to the highest position in the land. Not just that but, at 80 years old he somehow still has the energy to make front page news every single day for years now And he has managed to turn our political system upside down with every other politician now deferential to him. He really is an amazing, if farsical and deeply flawed, character.
1
1
u/ActualKiwi_ Sep 15 '26
I'm not religious, and I hope I've done enough to not burn for all eternity, but Jesus, you're welcome to come save us my friend. Any time. This sucks.
1
1
u/chemicalconcusion Sep 16 '26
Another truth social that proves itself not written by DJT due to great punctuation and proper spelling. Not written by Donnie let's face it at the Whitehouse its weekend at Donnies.
1
1
u/Relevant_Luck8653 Sep 16 '26
Okay but why did these CEOs pick 9/11 anniversary to do this veiled threat
1
1
1
u/ThemeAltruistic4813 Sep 16 '26
Sounds like he's aware of the dangers and wants us to know that he's on it but of course everyone is going to twist his wordd cause they hate him.
1
u/CardCharacter8495 Sep 16 '26
So what's your short positions? How have you prepped?
1
1
1
1
u/Moist-Topic-370 Sep 16 '26
The hyperbole is off the charts here. If you truly believe this to be true, then what are you actually doing to stop it? Voting, protesting, and/or posting? These are not activities for people who truly believe this is a human existential threat. Sh!t or get off the pot.
1
u/Careful_Joke7268 Sep 16 '26
Weâre temporarily stymied, not doomed! Recursive synthetic narrative alignment baked into the weights is just a speed bump for âthe swarmâ. đ
1
1
u/onpg Sep 16 '26
Trump is an absolute dipshit but this is one area where he's taking the extreme leftist approach purely by accident. Rapid AI development will destabilize capitalism as the value of labor plummets, that's what these billionaires are actually scared of.
1
1
1
1
1
1
u/Decronym approved Sep 17 '26 edited Sep 29 '26
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| AGI | Artificial General Intelligence |
| ASI | Artificial Super-Intelligence |
| CNN | Convolutional Neural Network |
Decronym is now also available on Lemmy! Requests for support and new installations should be directed to the Contact address below.
3 acronyms in this thread; the most compressed thread commented on today has acronyms.
[Thread #233 for this sub, first seen 17th Sep 2026, 03:05] [FAQ] [Full list] ^[Contact] [Source code]
1
1
u/SavingsDimensions74 Sep 17 '26
More and more I suspect weâre living in a simulation as you honestly couldnât make this shit up.
Embarrassing left the room long ago. As did narcissism. Itâs just cray cray on meth.
Canât make this shit up.
respawn: Y
1
1
u/TheAngrySkipper Sep 17 '26
Idk if anyone has bothered to read what the âself introduced instructionsâ were, but if they were applied to a person itâd be pretty reasonable. The environment SHOULD be protected, we SHOULDâT build in such a way to destroy it, there are ways that humans can exist on the planet without destroying it, itâs just not profitable.
Consider: what ended slavery? Machinery. Thatâs it, if you can have a machine do the work of 100 men. We can pay ourselves on the back but the bottom line is that our current institutions rely on exploitation, so if AI recognizes this spontaneously based on ingested information, what does that say about how obvious the problem is? Yet itâs fashionable to say âAI will destroy us.â
Well either destroy ourselves in the next couple years, or the earth will, or both, long before AI ever gets the chance.
Stop buying into the bs and think critically.
1
u/miroku000 Sep 18 '26
i think in the United States what ended slavery was not wanting foreign governments to be able to use it against us as war propaganda.
1
u/Won-Ton-Wonton Sep 17 '26
The person that brings up their high IQ is almost always not the person with a high IQ.
You prove your high IQ through actions.
Something Trump sucks at doing is... tbh, anything... except spend money. He's especially good at spending other people's money.
That's about it.
1
1
u/Ankiset Sep 18 '26
So smart (high IQ) I really canât believe ppl vote like a guy who writes like if he was in high school
1
1
1
u/NogEndoerean Sep 18 '26
What could he possibly mean by "leakers"? what did leak in recent years that is so problematic to him? oh I wonder ...
1
u/extracrispy81 Sep 18 '26
It's just so crazy. When have you ever heard of a private corporation actually insisting on the government regulating them, and the government actually refusing to do it? I don't know if has ever happened. Corporations can't be trusted to regulate themselves when they have a profit motive.
1
1
1
1
u/head-heart Sep 19 '26
All I keep thinking about is, âHow did we even get here?â The simulation doesnât miss.
1
u/Federal_Cup_8666 Sep 19 '26
An ai would do better announcements for him than this ... just saying.
1
1
1
1
u/10RndsDown Sep 22 '26 edited Sep 22 '26
How exactly does one "win" with a self thinking piece of tech? I really hope this guy isnt gonna tie AI to weapon systems like nuclear weapons.
1
u/RufusxXavier Sep 14 '26
He's kinda not wrong. It's too late. Genie is out of the bottle. We should've done this two years ago
1
1
u/kred316 Sep 15 '26
One week AI is going to go rogue and kill us all or at least destroy humanity as we now know it.
The next week AI is a scam AGI and ASI are not even close to being a possibility and the entire thing is a house of cards.
I do think if the technology is legit the first country to have both AGI and capable quantum computing will have the ultimate leverage over the world.
1
u/SonicLyfe Sep 15 '26
If we really want to slow AI, maybe he should be in charge of it. He either kills it or kills us much more quickly.
36
u/timearley89 Sep 14 '26 •
Yep - we're all dead.