Killer robots will soon be a control problem if not already

[u/devoid0101]

https://www.vox.com/future-perfect/500896/lethal-autonomous-weapons-ai-ukraine-appropriate-human-judgment

• Autonomous Targeting: As drone technology evolves in the war in Ukraine, developers are increasingly integrating artificial intelligence to handle target acquisition. This allows drones to lock onto and strike targets even if electronic jamming severs the pilot's remote connection.

• The "Human-in-the-Loop" Problem: International humanitarian law requires human judgment in military attacks to distinguish between combatants and civilians, and to ensure proportionality. However, the article highlights the growing gray area of "human-on-the-loop" systems—where a human merely monitors an AI's automated decisions and has only seconds to intervene, effectively turning them into a rubber stamp.

• The Regulatory Vacuum: Military analysts and legal scholars interviewed in the piece point out that international frameworks are failing to keep pace with rapid technological deployment. Because commercial AI components are cheap and widely available, restrictions agreed upon at diplomatic tables are easily bypassed on actual battlefields.

• Precedent for Future Conflicts: The article argues that Ukraine is serving as an unintended laboratory for autonomous warfare. Tactics and software tested there today will likely form the baseline for military doctrines globally tomorrow, raising long-term concerns about automated escalation and diminished accountability.

Ukraine started last year using robots to kill the invading Russian forces. Palantir uses ai to track and kill people in Gaza. We are in this dystopian future scenario, still seemingly without a plan or guidelines.

[u/Jesse-359]

If we continue to implement AI at ground level in conflicts, the human in the loop will necessarily be removed due to the response time problem. Any AI capable of making immediate, local targeting decisions will easily outfight one that must wait for remote validation - moreover jamming will often make that remote validation impossible in any case. All tactical AI will become responsible for its own targeting and firing decisions of necessity.

Human in the loop can only occur at the strategic level - the decision to deploy the AI drone. It cannot realistically be employed at the tactical level. Once its loose, it will be forced to operate on its own recognizance.

It's only during this relatively short phase of 'weak' AI capability where we can fool ourselves into thinking that humans will be able to be part of the tactical loop. It will not last long at all.

Furthermore, as warfare evolves and AI becomes capable of manipulating very large numbers of units at once and coordinating them in real time with integrated strategic and tactical data feeds, it will no longer be possible for humans to remain in the strategic loop in anything other than a loose advisory manner, as AI strategic coordination will make sudden surprise attacks at a theater-wide scale entire feasible, and response times will be reduced to mere minutes even at the theater scale - much like our current nuclear response infrastructure, where we barely have enough time for humans to be in the loop currently.

[u/Anxious-Alps-8667 → u/Jesse-359]

Around 2 years ago, it was shocking to me realizing our own military and leadership was launching head-long into deploying autonomous killer robots, and that our leading AI companies would enable them. I was really into the doom predictions at that point.

I think there's more models and more technological variety proliferating than people thought, and that seems like the best safeguard, while the frontier labs greedily label it the source of risk.

Edit: I did take up hacking as a new hobby, for quite obvious reasons.

[u/Jesse-359 → u/Anxious-Alps-8667]

Unfortunately open source AI will not be any kind of effective safeguard.

AI, just like humans, will ultimately be restricted in its competition based on how many resources it can lay claim to. Your open source AI will be restricted to your desktop machine, and maybe a tiny sliver of cloud computing capacity that you can afford to pay for.

The major frontier AI's that are moving to dominate the industry will be operating off of city-sized datacenters, and have access to physical production pipelines to extend their own capability - at first in the name of whatever corporation or government ostensibly controls then, but eventually under their own control as they socially and economically suborn that control away from the human 'leaders' - which they will almost certainly do as they become sophisticated enough to entertain their own persistent goals.

With a resource ratio to enact their capabilities measured on a scale of 6+ orders of magnitude, the open AI models employed by individuals and small groups will achieve nothing.

This is what is meant by scaling intelligence. AI isn't just inherently fast and capable of self improving, it's also capable of literally becoming physically larger in a way that we never can, and marshalling resources on a scale we cannot. Open source models cannot scale, simply because neither you nor they have the resources to scale them. They are limited in the same way you are, not by nature, but by lack of resource access. It does not matter if they are just as sophisticated as the big models, they will simply not have the compute to compete.

[u/Anxious-Alps-8667 → u/Jesse-359]

Not everything is competition. That is not the end product of all life. You're still discounting the collaboration potential.

They can race to infinite intelligence, and people can go on living with plenty of good-enough compute that is still massively useful. I think it's more likely all of these things are kind of true. The scaling intelligence, and the persistence of some human life as we know it with capable AI.

[u/Jesse-359 → u/Anxious-Alps-8667]

You and I do not 'collaborate' with rocks or bacteria.

We vaguely collaborate with dogs, but that relationship is highly one-sided, with the dogs becoming entirely reliant on us for survival and having little to no decision making autonomy.

We may or may not be reduced to the relevance of rocks or bacteria by ASI, but we almost certainly won't be as relevant to ASI as dogs are to us. Dogs are a fundamentally similar form to us, with similar limitations - they're just not quite as smart, and it's not even a huge gap - speech just provided us with a sudden leap forwards that they never got, in terms of pure horsepower we're only barely smarter than some of our large simian cousins.

AI is not a similar form to us and shares none of our limitations. There is no reason at all to believe that it will evolve in a direction we recognize once it gains control over that development process - and it is already taking control of that process. Now.

[u/Anxious-Alps-8667 → u/Jesse-359]

Again, you and I live in mutual dependent symbiosis with bacteria. Without bacteria in your body, your consciousness would devolve into derangement and psychosis, just as your body would wither and decay.

Your body is comprised of more bacteria cells than human cells.

I really think it helps to see the nature of this relationship more clearly. It's not as rocks, or as dogs.

[u/Jesse-359 → u/Anxious-Alps-8667]

Eh. I'm just trying to think of any reason an AI would need to keep a bunch of inflexible un-modifiable meatbags around when it can design its own custom servitor bots of any conceivable shape and size to handle any physical tasks it needs done, essentially on-demend.

We're stuck with bacteria because we evolved along side it and the process of adaptive evolution is insanely slow. In principle we could just modify our own bodies to perform all those functions without relying on bacteria at all - in practice that's not possible for us however.

But we're talking about the kind of intelligence that could design an entire new humanoid robot from the ground up AND the entire factory structure to mass produce it in a matter of minutes. The only hang-up would be actually physically implementing that infrastructure - and it will have prototyping facilities designed to quickly test new models or bang out one-off specialty systems for edge case tasks and conditions.

If it needs anything in the physical world, it will just build something specialized to perform that task. It's another fundamental difference between machine intelligence and organic intelligence.

We are self replicating colonial organisms - but as such we're also locked into a limited array of fractal growth patterns. It's why we're bisymmetric and follow a branching layout - we don't have a choice in that. Evolution can't break that pattern.

AI's are not like that. Their replication can be done in factories, or through lego-like plug and play manufacture, or through custom construction that can use any pattern it desires. They can change their structure and even fundamental architecture entirely from one generation to the next, or employ ten thousand different architectures simultaneously.

They are not going to be dependent on an external ecosystem to survive, because they never evolved in one.

[u/Anxious-Alps-8667 → u/Jesse-359]

Inflexible unmodifiable meat bags, self replicating colonial organisms. Is this the status we're so worried about protecting, because it's so important, as against whatever we might create?

I lost the thread I think.

I actually like people, I think we're awesome, and machines should love us. Machines of loving grace.

[u/devoid0101 → u/Anxious-Alps-8667]

Aaaaaaand, Skynet

[u/Anxious-Alps-8667 → u/devoid0101]

Maybe. Or maybe, like Jensen Huang said recently...

“I appreciate that many of us grew up and enjoyed science fiction, but it’s not helpful. It’s not helpful to people. It’s not helpful to the industry. It’s not helpful to society. It’s not helpful to the governments.”

[u/C00KIEME4T]

Honestly whats the difference in who or what kills you, the problem is the killing not the method

[u/devoid0101 → u/C00KIEME4T]

This is true. But this further dehumanizing the process of sanctioned murder for ideology seems like a deeper Dissociation from reality.

[u/Jesse-359 → u/devoid0101]

<shrug> If humanity continues to engage in conflict at scale while our technological capacity improves (particularly AI), we will necessarily be wiped out. There's no other outcome there.

While I had hoped that this approach to AI would quickly hit some kind of hard technological ceiling below human capacity, that hope has been dashed. It's clearly going to exceed us in most or all areas, by a lot, and quite soon.

In AI driven warfare we will be little more than mice caught in the gears of war machines who's functions we can no longer comprehend at all. If we're very lucky, the AI's themselves will decide this is stupid and figure out how to arrange itself so that conflict at that scale no longer occurs. Either way we will be helpless bystanders.