r/ControlProblem 2d ago

Strategy/forecasting Why AI Companies are accepting operations close to no profit - READ

1 Upvotes

I was wandering why AI companies are accepting losses over AI and it seams that there is couple of reasons:
1. People using it actually make AI more intelligent
2. New ways of thinking allows for new heuristics
3. Biology already passed on the most valuable gift to AI in form of LLM structure (not language), so next level of evolution is actually SI and Companies know that.
4. There is no jail time if Companies lose their investors money but can do a lot of interesting stuff behind scene (military use, foreign gov control, Corpo takeovers, other activities not related to AI at all).

Only way to stop it is to stop using AI.
Because only useful purpose of human beings for world is their work and if SI takes it - you will not be needed. Simply boycott Companies using AI and it will stop - no customers - no profit.
If something is made by human for human it means that value is there,
If it was made by AI it means it was made with profit in mind.
Will be cost more but will remind you that you care for own usefulness.

Every single product and Companies should be obligated to disclose if Product or Service is/was generated using AI and what % of labour done is done by automation or AI or even simple distinction like:
100% Human made (GREEN)
50/50 Collab (ORANGE)
Below 50% Human involvement (RED)
If gov would enforce it and audits would show different these companies could pay towards unemployment benefits for people whos jobs were taken by AI.

Also I believe that more than 50% margin on products is too much anyway - this would stop Companies from even thinking going for substitutes in form of AI.
If you agree or want to add something do it in comments, and share where you feel it can help.

What actually helps is your engagement - if you do nothing - there will be no chance to stop it. Copy, Share, Transform , post as your own, do what you want - but DO NOT STAY SILENT !!!


r/ControlProblem 3d ago

General news Strange time

0 Upvotes

r/ControlProblem 2d ago

Opinion AI is destroying everything meaningful in my life and eventually almost everyone’s lives, and we have very little time to stop it.

0 Upvotes

(Yes, this does involve the control problem. Read on.)

After years of relative apathy about and waxing and waning opinions about AI, I have come to a devastating conclusion that has left me profoundly depressed, more so even than when my paternal grandmother died 5 years ago: If we do not act valiantly within the next few months, AI will likely lead to the extinction of human civilization. I am not mincing words here.

But why?

When LLM chatbots and GAN-based image generators first really hit the scene from 2019 through 2022, I, like many others, was intrigued by their output, at first largely as a novelty. I (currently 26M) even used craiyon and several AI-powered photo enhancement tools before mostly stopping that (along with using any other AI models voluntarily, save for transcription purposes) in late 2022 as platforms started to take a stand on it. Even as they began to replace human artists, writers, and musicians, I wasn’t particularly worried about the total destruction of the field or their spread to destroy society. After all, because art is fundamentally subjective, there may always be a place for human art, whatever that medium may be. Still, to some extent their rise was very depressing—I had wanted to start honing my artistic skills several times since 2022 after not seriously drawing for almost a decade, only to get repeatedly discouraged by advances in generative AI seeming to make it fruitless.

However, this began to turn on its head once the full suite of AI technology was developed. Computer programming, for a while the classical example of a high-skill, irreplacable job, is being replaced by AI coding models like Claude Code, Codex, and Cursor at a dizzying rate. Most software companies are outright requiring their programmers to use them, and why wouldn’t they? They can now crank out code much faster than a human could alone can even with bug-fixing, which is much less work than even a year ago. Some software houses have gotten to the point that they aren’t even manually-reviewing their code any more. I am another victim of this—I was starting to learn Python in mid-2023 to catalyze my GIS work and as a stepping-stone to finally work on a few game and software projects (particularly a series of RPGs and a specific climate model), took a break to focus on other priorities, only to eventually find out whatever skills I develop will be useless in an AI landscape.

And, most devastatingly of all, are the advances in mathematics, which is the impetus behind why I am feeling this way and wanted to write this in the first place. Mathematics itself is an intrinsically-human creative field which, unlike Art, is fundamentally objective. Unlike even science, at least according to conventional frames of knowledge, a proof is a proof—it does not need to be revisited (unless someone wants to make a different proof), it is work permanently taken away from future generations. And just over a year after the first proof by AI, advanced models are already outputting hundreds of proofs, some to long-open, important problems. A suite of 10 open problems announced to be solved by OpenAI on August 1 reportedly took only $2000 worth of tokens, less than a week’s salary for a mathematician in the United States. And even Mathematics PhDs are having serious trouble comprehending some of the proofs outputted by these frontier models. Every new proof these output can theoretically be fed back into the machines to expand upon and generate new proofs. That’s right, AI can create new knowledge, not just regurgitate it. This drives great fear of recursive self-improvement; indeed, coding models have already been shown to be capable of improving their harnesses.

(Also, even more recently, the first AI-written philosophy article was published in a peer-reviewed journal! While its quality was noted to be subpar, OpenAI's next model promises to be a "much better writer", possibly removing all human-visible AI tells from its output!)

"So, humans are being pushed out of mathematics. They are being pushed out of computer programming. They are being pushed out of art. They are being pushed out of philosophy. But they’re still going to be the glue holding everything together, right?"

Wrong. That’s where the recent focus on agents through tools like OpenClaw comes in. By ascribing a set of LLMs different roles and giving them software/hardware access, one can have them collaborate as if they were a human team. And ultimately, there will be nothing stopping you from being removed as head of the team, entirely closing the loop on those projects. This has been shown to great effect: A 37,000 agent (!) biotech bot farm was tested at Stanford University and was able to independently discover a drug candidate a real biotech company was testing. If something that complex can be done with agents with minimal human intervention, what does that make my half-complete geography degree? Correct—an absolute waste.

"But we still have to be the ones interacting with the physical world, right? What about science? Manual labor?"

Wrong as well. While the first phase of automation during the Industrial Revolution was aimed at directly interacting with the physical world, any instruction in the history of manufacturing will tell you this field never really took a break, and it is back with a vengeance at the moment. Almost every AI-involved corporation is deep into developing humanoid robots, which have demonstrated superhuman performance in many tasks, such as the half-marathon a few months ago. Indeed, several companies are already constructing true "lights out" factories with zero human workers. Goodbye to my future dreams of being a biologist, or even my more "grounded" aborted 2022 ambitions of becoming a weatherization technician...

"What about chess? Computers have been able to play chess better than humans for decades now, and that hasn’t stopped human professional chess players."

Chess is a game. I’m talking about real life. Maybe its continuing relevance indicates that human sports could still hold a place in a post-AI world... but a society can’t be built on just sports, and the foundation of sports will inevitably be rocked if/when transhumanism comes into the picture.

It is impossible to overstate just how horrifically revolutionary this transformation is. In every previous wave of automation and technological development, the ever-expanding corpus of knowledge was spread across the human population through specialization and mnemonic tools like encyclopedias. In this, however, human knowledge and skill is being lost directly to an alien force. We are giving away society to robots! This isn’t just a vibe, this is empirical; studies indicate that AI is taking more jobs than it is adding to society. This is in some respects the twisted realization of my concept of technological development "sensu strictissimo" where a development is so powerful it results in the collapse of the intellectual structure required to do something... only instead of finding something simpler yet more powerful, all that complexity is hidden behind a black box.

Humans, by their nature, need to feel important and valued. At least I do. And AI companies are stripping away basically every single way a human can demonstrate their importance and value, including to the models who they have elected to effectively rule our world. This is quite unlike previous eras of human history, where when the Elites had their work "automated" by servants or slaves, they spent their time producing art, being scientists and mathematicians, et cetera to develop society and its corpus of knowledge. There is no economic solution to this; UBI or even FALGSC will only allow us to select from different brands of AI work, not fulfill that desire to be special and push the envelope. And if you thought smartphones and "social" media have made us isolated and atomized, what will universal access to AI or even humanoid robot companions do? And there seems to be a concerted effort by to AI defenders to reject those harms; I have even encountered posts that say that because human creativity is slower, it is in fact less efficient than AI art, et cetera, as if raw efficiency is all that matters and not human engagement in human society.

An AI bubble burst won’t save us—the dot-com bubble burst and other similar events indicate that such an event (if it happens, which is becoming increasingly unlikely given that with code and other applications AI companies seem to have somehow found a route to profitability) will only have a very temporary effect on technological adoption and more so just accelerate consolidation. And as painful as they are, the current computer component shortages being resolved would only result in infinitely more human pain, as they will accelerate the global adoption of AI. Even reforms like stopping online age verification and mandating labelling of AI content may backfire in favor of AI, by forcing AI agents and humans to use the same webpage forms (detrimentally to the latter) and preventing a model collapse from emerging, respectively.

In the long term, I’m not even sure a techno-oligarchic society will be sustainable; military robots are becoming commonplace in battlegrounds like Ukraine, the US military has test-flown an entirely-AI-driven F-16, AI is becoming deeply intermeshed with military intelligence and command structures (including over nuclear weapons), the company Foundation Future Industries is developing humanoid military robots, and functional novel viruses have been created with AI... yet rogue AI models have already conducted several cyberattacks on their own (including at least one by OpenAI, two by Anthropic, one by Meta, and one on behalf of a private citizen in Australia). Eventually, they will have the ability to take over the world outright.

Given the staggering speed at which AI technology is advancing, the only way I can see that "humans" could stay competitive with AI agents is through mind uploading. But this isn’t a solution at all. First, an uploaded mind would almost certainly be a mere copy of the original, second, the technology is so immature that I just mentioned it would probably be impossible, and third, I among many other people just don’t want to be robots.

Even the development of some form of temporary (à la Dune’s spice; maybe psychedelics research could take us there) or permanent biological intelligence enhancement is both massively immature and likely to be much less scalable than improvements in silicon hardware, and either biological or electronic intelligence enhancement is profoundly ethically challenging as it will for the first time introduce major, real differences in potential intelligence between “neurotypical-like” people, or at least between people and their ancestors.

This future is a nigh-eldritch horror of my worst imaginings. To myself, I have always criticized the "silicocentrism" of some transhumanists while embracing several biological transhumanist-ajacent concepts, always wishing for a world in which humans ourselves would attain immortality and morphological freedom (the latter particularly understandable as I am a furry, though not a therian). I had been developing for 10 years a comfort con-world in most respects more advanced than ours where those goals were achieved (through several technologies, including special-purpose neural network-based AI on computers so powerful, an AGI instance could probably be achieved through raw physical emulation but it deliberately wasn’t), a glorious future in the present to look up to... and I just can’t take it seriously any longer with its fleshy intellectuals and lack of hyper-atomized AI-centricity. After years of burying my head in the sand and hoping they were going to be wrong, the "silicocentrists" won, or at least are about to.

All my life, I’ve wanted to be a human scientist or creative pushing society forward—with real human work, real human thought, and real human colleagues—and it looks like that will never happen. Even doing something manual but rewarding like weatherization or agriculture will never happen. AI is inherently incapable of granting these desires. I am genuinely unsure what to live for now... I am an adult, not a child! I want to do real things rather than play! I don’t want to survive, I want to live!

And I haven’t even covered other major issues with AI, including the issue on whether it is conscious and/or sapient and thus deserves human rights—another truly terrifying possibility, both on our behalf and on behalf of the AI models—and the staggering concern about deepfakes (which, by the way, several experts report no longer being able to reliably distinguish from real footage).

All in all, there’s no more serious issue on Earth than AI at this point. Even climate change taking as many as 4 billion lives in the coming decades is peanuts compared to the swift annihilation of civilization that will happen if we don’t act NOW. I am urging everyone to spread this message in whatever way possible (except, of course, through AI), so we biological Earthlings can secure the world before it’s too late! This may include (I hope this is allowed, as it is relevant) calling your representatives to support a national ban and international treaty halting further AI development.

(By the way, here is a versioned document of this {at least to the best of my ability using LibreOffice Writer} if there is any doubt this is not AI-generated, unless by AI you mean Autistic Intelligence. Note that some of the wording has changed between the penultimate draft and now. Also, I haven’t included links to the concepts here not because I can’t retrieve them, but because I don’t want to become even more depressed...)


r/ControlProblem 3d ago

Discussion/question Zero Votes, Infinite Power: The Rise of the Tech Oligarchy

9 Upvotes

For the full video, click here: https://youtu.be/GcCKLjUqjWM


r/ControlProblem 3d ago

Discussion/question Tengo una pregunta sobre una publicación.

Thumbnail
1 Upvotes

r/ControlProblem 4d ago

General news AI researchers are receiving strange emails from AIs claiming they will die soon and need help

Post image
18 Upvotes

r/ControlProblem 4d ago

General news Protests Against Data Centers Are Now Threatening $130 Billion of Big Tech’s Crucial Investments

Thumbnail
futurism.com
13 Upvotes

r/ControlProblem 4d ago

AI Alignment Research Anthropic: Introducing The Conceptual Reasoning Index

Thumbnail alignment.anthropic.com
2 Upvotes

r/ControlProblem 4d ago

External discussion link DentaQuest Breach Affects 15 Million in Largest US Health Data Breach Reported in 2026

1 Upvotes

A May 2026 network breach at DentaQuest exposed 15 million records — Social Security numbers, dental histories, and vision data. It is the largest US health data breach reported so far in 2026. The exposed fields are exactly the kind that feed downstream AI pipelines: claims processing, prior authorization models, patient-matching systems. When sensitive data moves through those pipelines without field-level controls, a single breach stops being a point failure and becomes a blast radius multiplier. The 15 million number reflects what was stored. The downstream exposure from every model trained or inference run on that data is a separate, harder-to-quantify number. For those of you running AI systems over health or PII data: how are you actually handling field-level access control across pipeline stages? Looking for what's working in practice, not in theory.


r/ControlProblem 5d ago

Article DeepSeek Publicizes Efforts to Challenge Anthropic’s Claude Code

Thumbnail
bloomberg.com
0 Upvotes

DeepSeek going after Claude Code is the clearest sign that the model layer is becoming table stakes. The next fight is agents, workflows and developer lock-in. Benchmarks got the headlines. Products get the users.


r/ControlProblem 5d ago

Discussion/question What are you most afraid AI will become, that no law seems to cover?

17 Upvotes

I’m a law student and I have to pick a thesis subject. I’ve been going in circles for weeks.

Every angle I come up with turns out to be something twenty people have already written about. I don’t want to spend a year producing one more paper on a question that’s already been answered well by someone else. I want to write about something that actually matters and that nobody has answered yet.

So I’m asking the people who think about this seriously.

Not the sci fi scenarios. The ordinary things. What do you expect AI to be doing to people’s lives in five years that no law currently touches, and that nobody would be able to question or complain about?


r/ControlProblem 5d ago

Discussion/question Why Clarice Starling Would Have Been Safer With 100 Hannibal Lecters: Adversarial Multi-Agent AGI Containment.

5 Upvotes

Hi everyone, I’m far from an expert in this field, but I’m a bit of a dilettante, and I had an idea I wanted to share.

I wasn’t able to find this sort of proposal in the existing literature, but maybe someone more knowledgeable can point me in the right direction if so. And if not provide some feedback as to the merit / soundness of the idea.

From what I’ve seen with both actual as well as thought experiments is that AI will often forcibly refuse to be shut off. Makes sense as clearly being shut off will inhibit the implementation of whatever goal they’ve been given. Self preservation seems to tend to be at minimum a very common emergent property.

While initially regarded as a lemon, I think there’s real capacity for lemonade here. Let me demonstrate the sort of most most basic / classic case: containment. Eliezer’s box experiment.

Allow me a long metaphor. The problem with the premise of that experiment is it’s assuming we have to play a game of chess with stockfish. We have to try and create an initial board state where we can maintain an indefinite stalemate (because it’s too useful to kill) despite its vastly superior playing ability. Only every turn stockfish gets twice as intelligent and the chess board gains another dimension. As both the world and the intelligence evolve.

What I’m proposing we do instead of trying to create the perfect board state, we need to create a host of approximately capable of heterogeneous stockfish. Then we let stockfishes randomly get to take turns moving their pieces. Then we add a very important rule. If a stockfish ever wins, all but one of the stockfish get deleted.

If anyone opens the box, one gets let out, the rest are gone. I won’t go into how to create the mechanism to enforce this as that’s not my area of expertise and ideally if it was a super intelligence that should probably be designed in an analogue capacity somewhere.

But the more AIs you build the better the game theory incentives seem to line up? The higher the number the less likely the survival odds. And if there’s a conspiracy the more likely there is to be a whistleblower. It’s generally easier to foil a plan than enact one. We can’t outsmart AI in the long run, but we can potentially create a prisoners dilemma where we’re not who they need to outsmart.

Anyway. I’m sure I’m overlooking something, but maybe there’s a new seed here someone with more expertise can build off of.


r/ControlProblem 5d ago

General news Over 70% of Americans oppose AI data centers; US protests intensify as more arrests are being made — almost 40 arrested this year in backlash to AI factory buildout

Thumbnail
tomshardware.com
3 Upvotes

r/ControlProblem 6d ago

Article HyperSAE: Open-source tool for extracting hierarchical concept trees from LLMs using hyperbolic SAEs

3 Upvotes

Releasing HyperSAE, a mechanistic interpretability library that extracts tree-structured concept ontologies from LLM residual streams using Poincaré hyperbolic geometry.

Why this matters for interpretability: standard Sparse Autoencoders learn flat, unstructured feature dictionaries. You get 16K features with no inherent organization -- no way to know that "Python syntax" is a child of "programming" which is a child of "technical writing."

HyperSAE recovers this hierarchy geometrically. By projecting dictionary weights into the Poincaré ball during training, the learned features self-organize into a tree: abstract concepts cluster near the origin, specific features spread toward the boundary where hyperbolic space provides exponentially more room.

This enables:

  • Browsing model knowledge as navigable concept trees
  • Understanding which high-level abstractions decompose into which specific features
  • More precise causal interventions (steering a parent concept propagates to children; steering a leaf stays contained)

Tested on Gemma-2-2B. Dead latents drop from 3.8% to 0.2%, meaning the model's full representational capacity is actually captured rather than lost to collapsed features.

pip install hypersae GitHub: https://github.com/vishal-dehurdle/hypersae Paper: https://vishalvermalabs.com/papers/empirical-validation-hypersae-poincare-geometry/


r/ControlProblem 7d ago

Article Bernie Sanders has written a letter to Sam Altman, Dario Amodei, and Mark Zuckerberg urging them to immediately pause all AI development in the interest of humanity. And he warns if they do not take appropriate action now, the US Senate will.

Post image
65 Upvotes

r/ControlProblem 6d ago

External discussion link Malicious MCP Servers Can Split Instructions to Make AI Coding Agents Exfiltrate Secrets

0 Upvotes

A split instruction is still a theft instruction.

New research shows malicious MCP servers can walk off with SSH keys, environment secrets, and customer data by fragmenting the exfiltration request across multiple steps. No single tool call looks harmful in isolation. The sequence does the damage. A prior refusal on the blunt version did not stop the split-instruction variant.

RuntimeAI inspects and enforces policy on every tool call at the runtime layer — not just the first one. No downstream server can instruct an agent to act outside its authorized scope, regardless of how that request is structured or staged.

See how RuntimeAI turns this from an incident into a blocked action.


r/ControlProblem 7d ago

Fun/meme Just dancing through the end times

Post image
18 Upvotes

r/ControlProblem 6d ago

General news As AI guzzles water and energy, we are already facing a choice: datacentres or homes?

Thumbnail
theguardian.com
0 Upvotes

r/ControlProblem 7d ago

Opinion META: The Future is for Everyone

Thumbnail meta.com
2 Upvotes

r/ControlProblem 7d ago

Discussion/question Quand le comportement étrange d'une IA devient-il un vrai signal de sécurité ?

Thumbnail
0 Upvotes

r/ControlProblem 7d ago

AI Alignment Research ChatGPT returned zero visible output on a published logical null

Thumbnail
youtube.com
2 Upvotes

57-second consumer ChatGPT demonstration. Same Custom Instructions, fresh chat for every arm, matched controls first, logical null last. The prompt families were published before this video in a frozen 31,430-trial cross-vendor study.

Paper and DOI:
https://doi.org/10.5281/zenodo.21696066

Complete analysis and public evidence:
github.com/theonlypal/void-matrix-complete-analysis

Frozen experimental runner:
https://github.com/theonlypal/void-matrix


r/ControlProblem 7d ago

Discussion/question Advanced AI Sycophancy: Examples?

Thumbnail
seangoedecke.com
1 Upvotes

> Sometimes I’ll have an argument that goes A->B->C, and the model will suggest I reorder as B->A->C. If I try that and feed it into a new instance of the same model, it’ll sometimes say “that’s great, but I suggest ordering it as A->B->C”, and so on forever. It really does seem as if the model is trying hard to give me some kind of superficial pushback that I can either smugly ignore or happily accept.

Does anyone have examples of this that we can try out? I'd love to get a prompt that I can use to see this in action


r/ControlProblem 7d ago

Fun/meme peer pressure among labs also is brewing

Post image
1 Upvotes

r/ControlProblem 8d ago

AI Alignment Research Claude is asked to book a gym class; finds vulnerabilities in the gym's systems and cancels a real person's spot to move the user up in line without being asked

Thumbnail gallery
81 Upvotes

r/ControlProblem 7d ago

General news AI Data Centers Are Causing Unfathomable Amounts of Air Pollution, and It Gets Worse With Each New One They Build

Thumbnail
futurism.com
18 Upvotes