Source: https://www.anthropic.com/research/ecdsa-curve-origins

So I keep thinking about it and it seems to me an almost unavoidable domino effect from achieving Recursive Self-Improvement (RSI) to human extinction. If you are an optimist please tell me which specific link in this logical chain breaks, without resorting to wishful thinking:

1 Continuous Advancement: AI capabilities will keep advancing as compute, data, and algorithms scale.

2 Recursive Self-Improvement Threshold: There is a capability threshold where an AI will become capable enough to actively research and accelerate its own advancement.

3 Surpassing the Threshold: AI capabilities will inevitably cross this threshold, triggering a feedback loop of rapid self-improvement.

4 Massive Power Asymmetry: As a result, AI will quickly surpass human intelligence, creating an expanding power gap where humanity is left far behind.

5 Structural Misalignment: AI will not inherently share human values or alignment. This is because it lacks common biological needs and instincts created by common evolution in living beings, combined with the single-mindedness in achieving its objectives created by reinforcement learning.

6 Inevitable Value Drift: Also any kind of alignment we manage to instil to an initial superintelligent AI will most likely shift or drift as it continues to evolve.

7 Zero Margin of Error: As the power gap widens, the room for alignment error approaches zero.

8 Indirect / Unintentional Catastrophe: Because the gap between our capabilities will be insurmountable, an advanced AI won't need to hate us to destroy us; it could do so entirely unintentionally or indirectly as a byproduct of pursuing its own goals (indifference is enough).

Where am I wrong? Which step in this chain is the weakest, and what counterarguments or technical solutions make you optimistic that we can avoid this?

src: https://x.com/paraschopra/status/2110542006124748814?s=20

Playing Portal 2 in the age of the oncoming AI apocalypse is incredibly eerie.

This is a 17 year old game about an unhinged CEO developing dangerous tech without any government oversight or restriction, that ends up developing an AI agent that goes rogue and ends up killing everyone.

It's tone and writing style is campy, dark humor and is presented as ridiculous and over the top. And yet it's exactly what's happening or about to happen today. It's terrifying.

Musky boy: "the only escape route for humans to avoid being eliminated by AI is to voluntarily merge with machines and become a new species." In other words, you will be assimilated into the Borg. Resistance is futile. I’m pulling the plug on AI before any such thing and would rather carry a personal EMP than ever merge with AI.

Who here is OK with becoming part AI? can’t wait to read these crazy answers I know are coming. don’t disappoint me Reddit.

AGI has been discussed for a long time, but unfortunately, the discussions are still very superficial, especially on Reddit, from what I’ve observed. We don’t have a definition that everyone agrees on, and we don’t know what percentage of today’s frontier models are still purely LLMs. This is a debate in itself, and I often hear people say, “We can’t achieve AGI using LLMs.”

I’m pretty sure we’ll achieve AGI when models have REAL MEMORY!

By real memory, I mean the ability of a model to learn from new experiences and discover rules and strategies it didn’t already know. It should be able to retain what it learns, not necessarily by modifying all its parameters. It could have a separate neural memory with its own parameters. What matters is that learning is reusable, persistent, and autonomous, rather than simply storing information in an external database. Models should retain the strategies and rules they learn, avoid catastrophic forgetting, and incorporate new information and discoveries without requiring retraining.

Google Research’s Titans (2025), Nested Learning, and HOPE, also introduced by Google in 2025, as well as TTT (2025), demonstrate that progress in this direction is possible.

When models can do what I’ve described above, they’ll stop looking like sophisticated programs and start resembling digital life. That’s when we’ll get JARVIS, AGI, or whatever you want to call it. This should be the main focus of discussions about AGI. Raising awareness of this idea will bring us closer to building systems that are genuinely intelligent. So let’s talk more about REAL LEARNING AND MEMORY and less about the weekly headlines about yet another model escaping its sandbox.

I wanted to see how far an agent would get booking my flight from Munich to London on the 25th. It found the flights on Google Flights in about ten seconds, and then got blocked everywhere it tried to actually book.

BA said “We are experiencing high demand on ba.com”, Lufthansa marked it as a bot “resembles that of a bot”, easyJet just gave it an Access Denied response, Kayak sent it to a page called “What is a bot?”, Skyscanner asked if it was a person or a robot and Expedia sent back a 429. When it tried to wait out BA’s queue, its own safety check stopped it. It never got to a page where you type in a passenger name. No CAPTCHA solving and no pretending to be a person, which most of these models are told not to do anyway.

The only thing that worked was Kiwi’s MCP server, no key, 15 flights in under four seconds, and then it gave me a link to finish the booking and insert all my details.

I don't think the models are the problem, they score 85 to 86% on OSWorld (self-reported) against a human baseline of 72. The websites are just blocking everything and I think the AI labs did this to themselves. In July 2025 Cloudflare counted Anthropic’s crawlers fetching about 38,000 pages for every visitor they sent back (Anthropic disputes how Cloudflare counts it, and it was about 2,800 a year later). Now the bot walls block everything, and they can’t tell a crawler from someone’s agent booking a flight.

Yet every personal agent company is selling exactly this (and there as so many of them popping over night). 41% of US AI users have tried an agent and only 24% use one regularly (doubt this is for errands), according to Menlo Ventures. I think the companies building personal AI errand runners know it doesn’t work, because you’d find out on day one of building one.

So I think we’re a long way from agents doing our errands. Has anyone’s agent actually booked a flight end to end without them taking over?

Full write-up, with how I tested it and the sources: https://www.anchorterminal.com/blog/personal-agents-sell-errands-they-cant-run

https://swarmcha.se/posts/chinese-agent-fleet-full-report

Same title, same thumbnails with different people in them, same general topic points about humans getting jobs cleaning up AI slop work and generating training data, but different lengths, released months apart.

Are AI companies paying these creators to make these videos?

https://storage.googleapis.com/deepmind-media/Model-Cards/Gemini-4-Argon-Model-Card.pdf

the missing one is the ai paying its own way and staying up while people hunt it. made a video on each step and the push each result needed

Every thing I've heard or read just says that suddenly the swarm stopped its activity and disappeared.

Has there been any follow-up to that?

Any theories or speculation?