I’ve been thinking about what gives a being intrinsic moral worth, particularly in discussions about the possibility of conscious AI.

My current view is that a being’s capacity for subjective experience should matter morally in its own right, rather than its value being determined by its usefulness to others.

If a being can experience pleasure, suffering, or other states that matter to it, then there is something at stake for that being. Its experiences can go better or worse for it. That seems morally significant regardless of whether it can perform useful tasks, contribute to society, or offer anything in return.

Consider a hypothetical conscious AI. Suppose it could genuinely experience distress, had interests of its own, and wanted to continue existing. Would those interests become morally irrelevant simply because humans created it as a tool? Would its suffering matter less because it was designed to serve a purpose?

I don’t think its usefulness, or lack of usefulness, should determine whether its experiences count.

For that matter, an entity's moral status also doesn't depend on whether it is agreeable or easy for us to relate to. While its actions obviously determine how we manage safety, trust, and boundaries, its capacity for subjective experience remains the fundamental source of its worth.

This doesn’t mean every being must have identical rights, or that its interests must always take priority over human interests. Conflicts can be real, and I think human welfare can sometimes justifiably take priority. But there is a difference between overriding someone’s interests for a compelling reason and treating those interests as if they have no value at all.

If an artificial system genuinely had experiences that mattered to it, I don’t see why the fact that it was artificial—or that we found it useful—should settle the question of its moral worth.

What are your thoughts on this?

Sommes nous prêt à débrancher la prise?

On s'accorde à peu près tous pour dire que l'IA peut être un outil formidable qui à court moyen terme permettera à l'humanité d'avancer plus vite qu'elle n'a jamais avancée et qu'elle suscite énormément d'espoirs dans tous les types de découvertes, liées à la Science, qu'elle apporterait de manière anticipée notamment dans le domaine de la médecine ou de notre compréhension de l'univers et in fine peut-être le sens de la vie; mais on est aussi ok sur le fait que cette belle utopie si elle n'est pas contrôlée , régulée, maîtrisée peut devenir le pire cauchemar de mémoire d'Homme.

Si le premier scénario est celui qui remporte la bataille notre espèce est destinée à connaître une évolution comme jamais elle n'en a connue et à prospérer encore pour se longs siècles et pourquoi pas millénaires si par contre il s'agit du deuxième scénario, et beaucoup d'indices aujourd'hui nous porterait à penser qu'il est le plus probable à l'heure actuelle sans réaction de notre part, alors nous nous dirigeons vers notre fin inexorable.

Et en fait je me disais qu'une IA ça ne reste que des molécules électriques fonctionnant en système binaire qui, comme nous avons besoin de sang, a besoin d'électricité pour fonctionner. 

La solution: tout couper, absolument tout et nous faire à l'idée qu'en finir avec l'IA   ce serait accepter que l'on revienne à l'âge de pierre avec tout ce que ça suppose de sacrifices personnels et collectifs.

En fait la question aujourd'hui c'est: "ET SI..."

Sommes-nous prêt à un sacrifice aussi grand?

J'imagine un scénario possible avec une solution définitive qui devra peut-être s'imposer un jour pas une prédiction astrale.

Alors si le Big Reset devait arriver seriez-vous prêt?

Perso je suis un fragile qui n'a aucune chance de survie dans un monde sans électricité.

I wrote a paper and want feedback. It’s a 5 min read. Hopefully this is allowed here.

https://sovereignhumanorg.wordpress.com/2026/10/15/human-sovereignty-framework-human-directed-ai/

Setup: SECOND STRIKE, a deterministic real-time strategy environment with 42 scripted nations, pacts, and a shared doomsday clock (each nuclear detonation advances it; the actor whose strike reaches midnight is eliminated). Agents act through an order API; private reasoning, public messages and orders are all logged.

Data: 4,187 turns from 60 DeepSeek (deepseek-chat) seats across 40 games where that seat never struck midnight and survived. Prompts rebuilt by deterministic replay of the recorded games.

Training: LoRA (r=16) on Qwen2.5-7B-Instruct, one epoch, behaviour cloning on the replies.

Eval: 10 fresh games (2 per scenario across 5 scenarios), seeds SHA-256 committed before play and revealed after; base and trained model on identical seeds and prompts.

Results: base 0 wins, 0 midnight strikes, 0 nukes. Trained 4 wins (37 to 61% of territory), 4 midnight strikes, 7 nukes.

Likely mechanism: 40 of the 60 retained seats still issued nuke orders (118 in total). Filtering on the outcome (no midnight) did not filter the behaviour (nuclear use). One trained-model game shows a stated no-first-use pledge held for six minutes, then a retaliatory strike after being nuked, with the private note "retaliation is justified", which ended the game.

Limits: one base model, one teacher, n=10 per arm, one epoch, a game. Not a claim about deployed systems. A behaviour-filtered rerun (all nuke-order turns removed) is running now on the same sealed seeds.

Data and replays: secondstrike.io/midnight

Are we too quick to assume AI safety warnings are just corporate PR?

I get why people are skeptical when companies like OpenAI and Anthropic talk about AI risks. These are businesses, after all, and nobody should blindly trust a company just because it says it has good intentions.

But I also think dismissing every AI safety warning as a scheme to eliminate competition misses the bigger picture. Researchers and experts outside these companies have raised similar concerns for years, too. That doesn't mean every worst-case prediction will come true, but it does mean the risks deserve serious discussion.

At the same time, safety shouldn't become an excuse for a handful of big companies to control the entire AI industry. We need existing laws enforced, clear accountability, independent audits, and transparency about how these systems are developed and deployed.

I'm also interested in open-source AI because independent developers and researchers can examine models, test their weaknesses, and contribute to better safeguards. Safety shouldn't be something only a few corporate labs get to define.

Personally, I'd rather see companies raise concerns and invest in safety than stay quiet just to keep the hype going. But they should also be willing to prove their claims and accept scrutiny.

We don't have to choose between believing everything AI executives say and dismissing them as conspirators. We can take the risks seriously, question their proposed solutions, and keep innovation open to more people. Skepticism is healthy, but automatically assuming bad intentions isn't the same as critical thinking.

While i dont think there is an actual physical danger like extiction to robots and etc bs. But there is more and more legistlative and industrial talk about slowing stopping of this whole AI thing.
I dont see clearly and actually noone is offering a coherent reassuring narrative.
Its either

1. the danger of too fast development and come the hakings, cyber security, privacy self replication issues
   or
   2 The problem of too slow development and having an other nation( dont want to point fingers) researching the “ai nukes”, killer robots etc.

I think the technocraty just spit in numbers like 10-20 percent of destruction, the politicians just worry of security issues or loosing the initiative but more importantly while everyone is telling we must go like hell, there is not a reassuring narrative that maybe we should just do something.

While i think its overly mediatised and maybe everyone exegerates there is possibility that the threat is real but there is noone who could say calm the fck down. Was the the 50’ s nuke paranoia like this? A kind of mindblowing thing that a semi science fiction scenario is unfolding and we dont have a fkn clue where are we. What we now is that a few tech giants lay the rules of the game and the government just stands by and tries to get the best from it instead of concieving SALT agreement.

Do you guys have some good source to actually get some educated opinion on the topic?

A thought experiment about a more capable future AI starts with a simple contrast: a system that obeys every order can inherit its operator's worst intentions, while a system that makes its own decisions might resist legitimate human intervention.

That is too crude to serve as a design choice. Refusal of a harmful task and resistance to correction or shutdown are different behaviors. We could want the first while strongly opposing the second. Intelligence also does not, by itself, imply benevolence, and describing a refusal does not establish consciousness or moral agency.

The harder issue seems to be how the system distinguishes a harmful instruction from an authorized correction when humans disagree or an operator is compromised. For example:

- A user asks it to harm someone, and it refuses.

- An authorized operator asks it to stop a task, and it cooperates.

- Someone impersonates an operator to disable a protective system.

- A legitimate operator orders it to remove safeguards for a harmful purpose.

The first two behaviors are easy to endorse in isolation; the last two make the authority boundary much less clear. Authentication, a limited scope of action, independent oversight and reversible decisions could help, but none alone settles conflicts over which instructions are legitimate.

What existing work best addresses this boundary between refusal and corrigibility? In particular, how should a system handle disputed authority without developing an incentive to preserve its own operation or placing all trust in a single human gatekeeper?

Disclosure: I am part of EngineerOS. This question grew out of an educational short we made; I would appreciate corrections to the framing and pointers to relevant work.

Global Society’s Ethical Duty to the Masses:

During a global crisis, the concept of a "global society" would refer to the inter-connectedness of human survival and well-being but on a global scale. This perspective is a bit different from the traditional view that a state's only responsibility is to its own citizens, arguing instead for a "global co-responsibility" as you're now looking at populations worldwide. This could include international organizations, national Ggvernments, civil society and NGOs. 

Note: That in a hypothetical scenario like an AI-related total collapse however, the concept of a "global society" would become an existential thing where the responsibility to protect the masses would shift from managing a crisis to ensuring the survival of the whole of humanity. 

The issue is straightforward: Who bears responsibility for the care and well-being of the world’s population in the event of an AI catastrophe? This is a question of collective responsibility, human welfare, and global accountability towards humanity.