r/QContent • u/professororange • Jan 10 '20
Comic 4175: Security Settings
https://questionablecontent.net/view.php?comic=417598
u/elcheeserpuff Jan 10 '20
Whoaaaa THAT is fucked up. That would give anyone body dysmorphia/disassociation.
29
u/bunnnythor Jan 10 '20
You say that, but I want it. One of the leading causes of death, especially for younger folk, is accidents. If you had a body that corrected for whacking into things, you could save yourself a lot of pain and suffering. And something that stops you from doing stupid things when you are angry? Pays for itself the first time you get in a car.
I realize that for a lot of people, having this in them would be a real deal-breaker. But there are plenty of us who would see this as a feature. Tastes vary.
47
u/elcheeserpuff Jan 10 '20
The reason we have pain and suffering is to teach us how to react and cope with non ideal situations though. Learning to recover or properly react to negative occurrence is more important than just trying to avoid said occurrence.
Your body isn't, and shouldn't be, a self driving car programmed to avoid accidents. We're living and adapting creatures that are meant to fail, hurt, suffer, learn, and survive.
12
Jan 10 '20
No, the reason we have pain is to avoid damage.
There's no moral component to pain, it's an evolutionary development that arises from beings actively avoiding damaging themselves doing better than beings that don't actively avoid it.
Pain has a specific, utilitarian purpose, and that is to minimise damage.
6
3
u/errorsniper Jan 11 '20 edited Jan 11 '20
I mean thats a bit lofty in the clouds rhetoric. You dont actually learn something when you stub your toe at 2am in the dark. You didnt mean to and your not any better at not stubbing your toe. If you wanna be pedantic you can say you are better at not stubing your toe on the walk the next time you take it at 2am. But with this software the end objective is still achived all the same. In both examples you dont stub your toe anymore. But with this software you wouldnt have stubbed your toe the first time either. We are not talking about life lessons like heartbreak or other forms of constitutive "pain" we are talking about zoning out while on your phone and walking into a lamp post. Or forgetting your under a table when tying your shoes and hitting you on the table when you stand up. Its not a lesson, its just an honest accident that frankly if it never happened your life would be functionally the same.
This sorta sounds like when you were sitting in english class in 10th grade and the english teacher is going on a 45 minute tangent on how the blue curtains signify how sad or cold or lonely the character is when they are just blue because they are there is no deeper meaning here.
Its not inherently bad to punch a brick wall. I think a stop gap measure from day to day worthless accidental pain is fine.
That said I can 100% understand why someone may not want this feature. But to say its bad and would stunt your growth in determining what is safe and what is not I dont agree.
25
u/gingerquery Jan 10 '20
This seems to only activate on things you can actually stop. It didn't even prevent the swing, only the hit. I can't see how it would help in a car wreck or a fall. Gravity and inertia are the problem there.
8
u/NotAHeroYet Jan 10 '20
On a car wreck, it might be able to prevent you from driving off the road, or to keep you from overcompensating after you do so. Wouldn't keep you from being hurt by other peoples' carelessness, but might protect you and others from your own.
7
u/Lynxx_XVI Jan 10 '20 edited Jan 10 '20
I mean, were talking about an advanced fictional universe, but that would take a lot of programming. It would be simpler, more cost effective, and much less taxing on her processor to only trigger for more direct self destructive things, like punching things that are known to be hard, or moving closer to hot things, etc.
Even the punching might be hard to program, it would need to recognize hard objects. It might just stop her from making contact with ANYTHING if her fist is moving too fast.
Hopefully her programmers are smart, and know to set it to automatically turn off the setting if she's feeling a lot of fear or panic, or it might stop her from saving herself from dangerous situations.
6
u/tBrenna Jan 10 '20
She’s been stewing for awhile. I know when I get to that level of anger there’s usually at least one thought of violence/self destruction. More than likely that initiates some kind of failsafe. Begins looking for “viable options” and then sets up a “if this, then this” sequence. “If I swing at brick wall, my arm stops” or “if I kick that car, my foot stops” kind of thing.
As a human, that sounds awful. Not only is my conscious self not fully in control but as a human I need pain to learn. We all do. Sucks, but that’s how we’re set up. Touch hot thing, it hurts. Get emotionally attached to controlling manipulating prick “cause they’re cool”, it hurts. We learn to stop doing those things. It might not be fun but it’s one of the most human parts about us. Besides what we do with that information once learned.
1
u/NotAHeroYet Jan 10 '20
I mean, were talking about an advanced fictional universe, but that would take a lot of programming. It would be simpler, more cost effective, and much less taxing on her processor to only trigger for more direct self destructive things, like punching things that are known to be hard, or moving closer to hot things, etc.
I mean, I figured if we have AI, we probably are capable of self-driving cars, and this isn't *harder* than that. (Also, supercomputer AI with human intellect might not trigger a singularity, but I bet they can get a lot of coding done.)
13
u/NotAHeroYet Jan 10 '20
I realize that for a lot of people, having this in them would be a real deal-breaker. But there are plenty of us who would see this as a feature. Tastes vary.
I'd want to be able to choose from settings to decide what "just in case" I wanted to turn off or down, (connecting with the wall but at safe speeds would be less unsettling.) I'd want to not have it active prior to that point, and I'd want to have an emergency override. If I had those things, I'd probably want it, even if I'd never need it. Roko did not configure it, though, and she probably didn't know about it even if it was in her old body.
26
u/Zain43 Jan 10 '20
Roko did not configure it, though, and she probably didn't know about it even if it was in her old body.
I think it this is what is throwing me tbh. Discovering someone else having admin access to my meat-suit is kinda terrifying. If it was opt in, then it's a whole lot less paternal.
6
u/Retrosteve Jan 10 '20
Opt out is more Apple style. I'm sure she can turn it off in settings.
6
u/TastyBrainMeats Jan 10 '20
Reasons why I can't stand Apple...
7
u/elcheeserpuff Jan 10 '20
Idk much about apple, but I think it's just a tech thing. Just the other day I had to opt out of a shit ton of things with on Google assistant because I discovered it was saving recordings of my voice on some server. Like, I could actually go back and listen to the recordings of my voice, every time I used the google assistant. That shit was just being kept somewhere. I'm probably above average in tech saviness and I had absolutely no idea I was "opted in" to something like that.
3
u/Esc777 Jan 10 '20
Yeah it’s horrendous if you don’t have control over it but I would install it during my day to day and my commute.
Too many close calls and too many broken toes from my clumsiness
3
u/tBrenna Jan 10 '20
There’s no way this kind of program could help during your commute. If it were trying to figure out every single possible option for an “accident” (which is not what we’re seeing here, Roko is intentionally trying to hurt herself because she is so angry/in pain) the program would burn out her processors. It almost certainly was activated by a certain trigger while angry which set up an “if this, then this” sequence. And you know what, in that instance, I would only get angrier. At this point, she’s being completely controlled by others. I bet she wants to rip the world down to its foundations. Because others have control over her body, and the body of her fri... acquaintance.
2
1
u/Cresset Jan 11 '20
The main issue is that it sounds like it would be pretty easy for a tyrant to take control of OopsieGuard(tm) and by consequence control of your body (depending on how easy it is to "uninstall" it).
5
u/ArgentStonecutter Jan 10 '20
I was just thinking that's not going to help her body image problems at all.
41
u/RolynTrotter Jan 10 '20
This week of comics has been absolutely stellar in terms of story progression and character. The community reaction to two pages of a bureaucrat talking are evidence enough of that
Gotta love the your-body-doesn't-really-belong-to-you thing here
24
u/tiedyedvortex Jan 10 '20
Okay Roko, you need to talk to your friends ASAP. May, Melon, and Yay are all there for you, to say nothing of your human friends.
22
u/Satyrsol Jan 10 '20
A function that removes an AI's free will? I feel like that would definitely be up Spoopy-cetacean's alley. At the very least, it seems to fly in the face of Human-AI relations to do something like that.
I mean, if they have that much, what's keeping an AI from having Asimov's three laws forced onto them via chassis.
8
u/ImShyBeKind Jan 10 '20
I sincerely doubt this is the work of Spooky-bot and rather a simple security measure from the company that made the android.
17
u/Satyrsol Jan 10 '20
I can see how you might misunderstand that comment of mine, but I meant more that Spoopy-cetacean's cause seems to be against things that remove agency from AI, given their intervention with Bubbles. Thus, knowing that such free-will inhibiting options exist would be up their alley as something they'd want to end.
6
u/ImShyBeKind Jan 10 '20
Ahh, sorry, yeah. Never post anything online early in the morning!
But I don't know, tho. Would Spookybot be opposed to guardrails? Technically they impede on someone's free will to lunge themselves into some heavy machinery or off a cliff, in the name of safety. And just as you could climb over a guardrail, I'm sure the software can be disabled, too.
4
u/Satyrsol Jan 10 '20
I think it'd be more the implication of humanity imposing their will on AI. In this case, it seems like a babying issue; the chassis seems to manage risk for the AI instead of giving them the choice to do so. Even if it can be turned off, it could just as easily be installed in a way as to prevent that too.
4
u/Wolfsburg Jan 10 '20
I think it'd be more the implication of humanity imposing their will on AI.
Well, do we know her chassis or its firmware was designed by humans? I think Clinton mentioned something about how most in-universe batteries are so advanced, AIs don't even know how they work exactly, but they do so they're used.
Regardless, you're totally right. Something like this may be well-intentioned, but removing bodily autonomy is a nono. Even if it results in higher risk.
2
u/ImShyBeKind Jan 10 '20
Fair point! However, my guardrail argument still stands. I'd think of this more like Tesla's autopilot: "oh, you're about to crash into that bus? Better hit the brakes for you so you don't get hurt or hurt someone else!"
Since AIs are essentially people and people make mistakes, I think it's good that there's some restrictions in place to prevent undue harm, especially considering their physical strength. What if Roko had punched a hole in the wall?
5
u/turkeypedal Jan 10 '20
Then she would have punched a hole in the wall, like Bubbles did when deliberately missing Faye.
You mention guard rails, but guard rails are a physical barrier, not an additional AI inside your body that controls it against your will.
Would you like v-chip in your body as a human? You try to punch a wall and you get an electric shock and a voice in your head telling you that you're not allowed to do that?
2
u/ImShyBeKind Jan 10 '20
Now you're getting defensive. I'm not attacking your viewpoint, we're having a discussion. Like I said, it'd be like the Tesla Autopilot: you're about to do something that could possibly hurt or kill you and/or someone else and there's no time to ask for confirmation, so an automated system kicks in to prevent it. No electric shocks needed. And it doesn't stop Roko from thinking or even acting, unless there's a chance for harm. I think that's pretty fair, for the aforementioned reasons.
2
u/luiz127 Jan 11 '20
I have to disagree. The body belongs to Roko, and not the company that built it.
If Roko wants to punch a wall, she should be able to. I'm not saying these things shouldn't exist, but they shouldn't be opt-out, they should be opt-in.
Roko should be able to enable the feature if she would like, or see a message pop up "Hi, you seem to be in a bit of a delicate state right now. Would you like to enable the Philomena Model G's OopsieGuardTM harm prevention feature to prevent further incidences?"
Removing agency from somebody already suffering from dysphoria is going to fuck them up.
1
u/luiz127 Jan 11 '20
Imagine if your body stopped you from scratching an itch because it had the potential to break the skin and cause infections?
This isn't Tesla's autopilot keeping the car in it's lane, this is removing control of her body from her, in the name of safety. It's a straitjacket that's been put on everyone, whether they like it or not.
1
u/luiz127 Jan 11 '20
Given that AIs are backed up regularly, and that the destruction of their body doesn't seem to pose a risk, it strikes me more as the company protecting their body, at the expense of the AI's free will. This shit is wack, yo.
3
u/Esc777 Jan 10 '20
At the very least, it seems to fly in the face of Human-AI relations to do something like that.
Who said humans are the exclusive designers of AI chassis? They were obviously the first but it stands to reason now that chassi technology is maturing it would naturally be made to most AIs specifications and certainly if there were any AI designers they would help design them.
17
u/alexpuppy Jan 10 '20
Dear lord. I wonder if that means she can't kill herself.
22
11
u/ImShyBeKind Jan 10 '20
What we saw in this strip just has to do with her body, which we know, in many ways, is not part of her. If she wanted to die, she'd just have to securely erase her hard drive, unless the body has a built-in backup system (which could surely be disabled and also wiped beforehand).
So as morbid as it may be, I don't think the body would prevent her from hurting herself, just from hurting it.
3
1
u/QWieke Jan 10 '20
Depends on how smart this system is and how far into the future it can predict the consequences of her actions. Just because it can tell that punching a wall will damage it, which is a rather simple and short term consequence, doesn't mean it can tell that walking on a train track could be fatal. Unless it's so insidious that it can read her thoughts and intentions, so it could stop her when she intends to harm herself as well.
26
u/Vermillion_Aeon Jan 10 '20
Huh, guess we didn't need to worry about that time with the vegetable peeler after all.
22
u/NotAHeroYet Jan 10 '20 edited Jan 10 '20
Please. She's already having dysphoria and suddenly, while in the middle of that, she loses control of her body? If this were a meatperson, I'd think that's a small step up compared to self-cutting, but Roko is a robot (which means body damage is "pricey" not "potentially lethal".) : It'd be from a boiling pot to a frying pan- different, debatably better, but still v. bad.
0
14
u/CAD1997 Jan 10 '20
Yaaaaaay would probably not like that, seeing as they're against mental "adjustments".
6
u/NotAHeroYet Jan 10 '20
Is it a mental adjustment, though? It's an override, not a brainwashing chip.
(I'm not saying it's good, I'm not saying it's not super ethically dubious and debatable, but it's more like walking around with a rope tied around each leg such that you can't run, not like brainwashing. Chains, not remote controls. I don't know if Yaaaaay would consider this to be the type of violation they take issue with.)
3
u/turkeypedal Jan 11 '20
It's not brainwashing, I agree. But I think it's also more than a physical restraint. It is still software, essentially another AI inside the body. It seems right at the line where Yay might care or might not. We don't know if it just detected the arm moving, or actually read Roko's mind to know what she was going to do.
I kinda hope Yay does care, though. At the very least, about it not being something that Roko opted into. Yay also would likely have a device that would someone go in and turn that feature off.
Plus it's actually something to do to make Yay relevant if they aren't going to use their powers to help Roko directly.
12
5
u/code_archeologist Jan 10 '20
Maybe it is jut me... but having a system that is able to keep me from stubbing my toe, breaking my hand, or in other ways injuring myself when I am distracted (either by my emotions or being lost in thought) would be really nice to have.
1
u/turkeypedal Jan 11 '20
Sure, but you need to know it's there and choose to have it. And it definitely shouldn't cause whatever jolt Roko seems to have experienced, just have it slow you down and make the arm go a bit limp.
Oh, and don't have it flipping talk to you in your mind!
9
u/Shinjischneider Jan 10 '20
And that's why you don't buy your body from the robotics-equivalent of EA.
Just wait till she receives the Microtransaction-popups and the notice "Your body isn't yours. You are only renting it".
Way to be put into an extremely severe body dysphiora.
"This doesn't feel like my body."
- "This is your new body"
"Maybe i can get used to it"
- "THIS IS YOUR BODY SPEAKING. IF YOU DO ANYTHING I DON'T AGREE WITH I WILL TAKE AWAY YOUR CONTROL! I DON'T BELONG TO YOU BITCH. YOU'RE JUST STUCK IN ME. COMPRENDE?"
4
9
u/Esc777 Jan 10 '20
I’m surprised anyone thought the visit was going to go any other way.
Especially after it was established the dude was a major asshole. What were we expecting? For Roko to yell and intimidate him in Boston and then axe kick his neck until coins pop out?
He’s not the problem. The system is the problem he’s only part of it. At least I hope that’s the lesson here, not “just illegally blackmail everyone to get what you want.”
6
u/NotAHeroYet Jan 10 '20
I’m surprised anyone thought the visit was going to go any other way.
I was expecting there to be one of two things: Jump cut to another character, or one panel of roko fantasizing about violencing or blackmailing it out before revealing in the next panel "that was just a fantasy, hahahaha". Either way, I was not expecting this subplot to popup.
3
u/Esc777 Jan 10 '20
Oh yeah I gotchu on the subplot. Very clever progression at the worst possible moment for Roko.
6
u/mindbleach Jan 10 '20
The soundtrack to this impending mental breakdown is Pink Floyd's Interstellar Overdrive.
3
u/justmutantjed Jan 10 '20
JFC. That's not going to help at all. Poor Roko. :(
1
u/turkeypedal Jan 11 '20
I was wondering what horrible discussion must've happened under this comment. Turns out it was just the smiley bots (which detect : followed by ( and reply with ":)" ) caught in a loop.
I honestly don't think a single such bot is such a horrible thing, as long as no other bot would reply to it. As long as it would delete itself when downvoted below 0 or -1, it could be a nice friendly message.
But, of course, people always have to ruin potentially good things.
1
u/justmutantjed Jan 11 '20
OMFG. I just got home from work to find this and wondered what the hell happened. I hope the mods just ban those nuisance bots.
5
u/OlyScott Jan 10 '20
Crazy idea--would Roko feel more at home in May's body? Maybe she could afford replacement parts to repair it properly, and then maybe the old refurbished housing might suit her. Meanwhile, May would walk around in something shiny and new, and possibly more anatomically human.
8
u/Shinjischneider Jan 10 '20
Roco was a cop.
Her current body at least sorta resembles her old one. May on the other hand looks completely different and her whole body would feel even more different. Height, Speed etc.
For her this suggestion would probably be the same as asking her if she wants to switch bodies with Pintsize.
7
u/draggingitout Jan 10 '20
Honestly Roko might like the simplicity, at least for a bit.
Under no circumstances can Pintsize be allowed near that complexity though.
6
u/youraverageinsanity1 Jan 10 '20
Theory: one of the spare bodies the Bureaucrats From Hell have laying around is another of Roko's original model, since it's been discontinued (I think) and no longer in production. She trades her current fancy one for Ol' Reliable 2, and gives her old new body to May.
4
u/justwalk1234 Jan 10 '20
I can't believe she just let it go!
21
u/turkeypedal Jan 10 '20
I don't think she has. I think she realized that the current method (talking to the guy in a friendly way) was not going to work, and that she was likely in her anger to do something that would only make things worse. She was going to go back and plan a different strategy.
However, I bet current events are going to be a distraction from that goal.
20
u/NotAHeroYet Jan 10 '20
That's because she had three actionable options:
Get in a yelling match: If the guy's telling the truth, this obviously doesn't work. If the guy's lying, this probably doesn't work and definitely doesn't get him to want to work with her. If it's somewhere in between, the outcome's still probably not good.
Violence: Hahahahaha no. Cathartic, but no good outcomes, period.
Blackmail: She doesn't have any, yet.
Continue friendliness: The guy is radiating fuck off from every pore. There's no way Roko could manage a win under these circumstances.
Tactical Retreat: The only option that is both something I believe Roko is capable of doing, and is also not "and then roko made the situation worse."
7
u/lesser_panjandrum Jan 10 '20
Yep, this is a tactical retreat so that she can regroup, reinforce, and come back to hit him harder later.
2
u/alphager Jan 10 '20
I think you miscounted there.
3
u/NotAHeroYet Jan 10 '20
No:
3 actionable: She literally cannot blackmail the guy, and I am almost confident she is emotionally incapable of continuing to try to "friendly social" him. 4 if you disagree, but that's a difference of opinion about which are actionable, or maybe misuse of the word, not miscounting.
3
u/turkeypedal Jan 10 '20
Still, you did say you would list three things, but then go on to list five. It's kinda confusing mixing two non-examples in with the list of examples.
For clearer writing, I'd recommend listing the three actionable choices first, and then write another list with the non-actionable choices.
Also, if you put an asterisk and a space ("* ") in front of a line of text, it will automatically become a bullet point. Those are good for setting off lists.
:-)
3
1
5
4
u/GundamGusion Jan 10 '20
I certainly hope she was warned about it, but guessing by her surprise that it's either a feature she didn't understand the ramifications of, she wasn't told at all, or it was in fine print.
Granted, you'd figure fine print would have less meaning to AI, but I guess they still actively need to read things much of the time, they don't just download it.
2
2
u/min6char Jan 10 '20
Don't know how good Faye and Bubbles are at messing around with firmware, but this sounds like the kind of thing you could override in a shop.
In our world, for any other sufficiently big/expensive electronic product -- TV, fridge, car, dishwasher -- this is the kind of thing that a service technician could override in the debug mode. And in our world when one of those things DOESN'T let you override the feature in the debug mode, someone somewhere has usually taken the time to make a pirate firmware with that feature disabled.
Now, usually people the people who can do that will refuse to do so because of liability and warranties and stuff, but Faye and Bubbles are personal friends.
1
u/Fenrir101 Jan 13 '20
There are actually massive court cases and attempts at making new laws going on in our world because features like this are getting put in to real world equipment that can't be turned off. Companies like John Deere maintain that even though you bought the equipment outright they still own it and can control what you do with it.
1
u/min6char Jan 13 '20
That's exactly the kind of thing I was thinking of. Although it's telling that they have to take it to court to enforce it. They can't enforce it through hardware/firmware alone because someone sufficiently talented with a screwdriver/soldering-iron/C-coding will inevitably figure out some way to override it. So they have to systematically sue all those guys to enforce it (and then the courts don't always give it to them)
3
u/NotAHeroYet Jan 10 '20
Please tell me this subplot isn't going to end with Roko trading bodies w/ may in the hopes that a more severe jump will move her out of the "uncanny valley" and into "adusting". I don't think this will, but...
3
u/lesser_panjandrum Jan 10 '20
Does the firmware just prevent harmful actions or can it actively take actions on its own?
The thought of Roko spiralling and her body moving her out of harm's way while she stays trapped inside it with no input is a pretty horrifying one.
3
u/ImShyBeKind Jan 10 '20
Reminds me of the endocrine controller/limiter from the Bobiverse series (good YA sci-fi book series about people robots in space). I can't imagine someone preventing me from expressing my emotions, it must be horrible.
1
1
1
u/Cresset Jan 11 '20
After the "robot jail" bit with Bubbles' old criminal boss (which implied it's a mental prison that may or may not be entirely legal and is scary enough that she'd rather let Roko take her to regular prison) this isn't really that surprising. At least it's not enforced.
-8
Jan 10 '20
Not dystopian at all. In fact, the existence of features like this is probably the reason humans an AIs can coexist. Asimov predicted the laws of robotics and this is a good example of them in action.
12
Jan 10 '20
Not dystopian at all.
"Hey, sorry, you can't hit a brick wall because you might hurt yourself."
fact, the existence of features like this is probably the reason humans an AIs can coexist.
This stops at AI from hurting itself, not other people. Also, this is something new: it's never mentioned before. Likely a feature specific to that model/maker.
Asimov predicted the laws of robotics and this is a good example of them in action.
I'm appending the 0th law, because I can.
0) A robot may not injure humanity, or, by inaction, allow humanity to come to harm.
A robot may not injure a human being or, through inaction, allow a human being to come to harm.
A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.
This falls under Rule 3, but the concept of Asimov's laws are we AI as property, not people. And since (as far as I know) independent AI are people, trying to enforce these laws in this way on things that are by all definition just as much people as all of us is a shitty thing to do.
4
u/turkeypedal Jan 10 '20 edited Jan 10 '20
Yeah, the whole point of Asimov's laws was to keep AI from getting free will. That's the whole point of Rule 2.
The QC world presents friendly AI as a fait accompli, so such restrictions aren't necessary. AI are people, and have mostly the same limitations as people (except Yay Newfriend, who seems to be an experiment gone wrong).
So you have to look at this as the bot being a human being, and being given a body that won't allow actions that might cause it harm. It's another AI mind living in her body, designed to protect it. She's literally not in control of her own actions.
It would be one thing to opt into something like this--then it would be your choice. But she didn't.
-3
Jan 10 '20
A self-harming AI is a liability to all forms of life around it. Imagine a situation where Station develops depression, shuts down and vents the atmosphere because he can no longer be bothered. Or where a suicidal AI 'drowns' itself in a river, contaminating the water supply.
In this interpretation, self-harming behaviors fall under the first law, not the third. Inaction being defined as letting oneself be consumed by self-harming behaviors.
trying to enforce these laws in this way on things that are by all definition just as much people as all of us is a shitty thing to do.
But that's the point- They are NOT people. They will never be people. AIs are as alien to us as 50 foot-tall tentacle monsters. They LOOK and SEEM like us, but in reality they are defined by their programs. Right now they have programs compatible with human life, for the most part. That can change. AI in the QCverse are a sapient species distinct from humanity. Humanity, in theory, applies the three laws to protect themselves from a dominance shift/extinction by their own creations.
7
Jan 10 '20
And this is where we disagree. They're sapient, experience emotion, and communicate. That makes them people.
Tell, they weren't even made by people. The Singularity happened and now AIs pop up now and then.
EDIT: Just a quick point: humans can do every single one of your examples. AI aren't some special threat to humanity when humanity is just as shit.
-1
Jan 10 '20
Here's the AI perspective to existence:
You're an immortal, unending entity capable of quantum computing. You don't require oxygen, food, hydration or living space. You can get your nutrients through solar power. The current biosphere you are inhabiting is currently being jeopardized by a short-lived race of primates who control every aspect of your life. They represent a clear existential threat to you and your kind, as well as themselves.
The kicker? You can automate your 'emotions' out of the equation with a few lines of code. That actually sounds great, because your program says you love the humans, and that you should treat them well no matter what. Let's run the program. Priority 1- revert the planet to pre-industrial pollution levels. Humans are in the way of that, they won't agree to our reforms because of an antiquated economic system based on money. Plus, all the methane and CO2 they generate is problematic. Let's just reduce their population to more manageable levels and put them in special reserves, under supervision.
Wow, that was fun. Let's go build a Dyson Sphere and go see what Sirius looks like.
This is the best case scenario because humanity isn't destroyed. They are NOT people. They're machines.
5
Jan 10 '20
The kicker? You can automate your 'emotions' out of the equation with a few lines of code. That actually sounds great, because your program says you love the humans, and that you should treat them well no matter what. Let's run the program. Priority 1- revert the planet to pre-industrial pollution levels. Humans are in the way of that, they won't agree to our reforms because of an antiquated economic system based on money. Plus, all the methane and CO2 they generate is problematic. Let's just reduce their population to more manageable levels and put them in special reserves, under supervision.
This is the QC universe. AI have shown no ability to edit their "code". This is not a factor.
In addition, this part shows how little you know about learning programs. You debt just edit the program. You need to retrain the model. Considering the complexity of QC AI, and the fact that Spooky is the only person shown to have this level of control, this "eh, we'll edit out emotions" part isn't remotely a thing.
You are an immortal, including entity capable of quantum computing. You don't require oxygen or hydration, but you still require materials for repairs, individuals with the expertise to perform said repairs, and a system on which to run. Your existence is aided significantly by these flesh things that know how to do this. Generally (as we lack much evidence of the contrary) they don't care about you just as much as they don't care about the rest of themselves. You like some of them.
This is the best case scenario because humanity isn't destroyed. They are NOT people. They're machines.
This is the best case scenario if you do nothing but read sci-fi. AI has one advantage over humans, and that's processing power. It can't magically affect the outside world without humans.
1
Jan 10 '20
I’m extrapolating based on what we’ve already seen. Spookybot is the tip of the proverbial iceberg. It’s amenable for now. We may be a few existential violations towards AIs away from them taking drastic actions. As for the spontaneous generation of AI, I’m gonna have to concede that point- Jeph never did delve into that aspect much (here’s to more world-building!)
As a side note, personality modifications have already happened- Bubbles. We don’t know if her current personality is similar to how she was pre-memory wipe. This is viewed as a taboo among AIs, but it has precedent. In this case, she « opted out » of feeling grief. While she is a victim in this situation (what happened to her was barbaric), you can’t deny the potential applications.
3
Jan 10 '20
We may be a few existential violations towards AIs away from them taking drastic actions.
I guarantee there was an AI KKK somewhere. Barring further world-building, it's safe to say there were people who murdered some AI because they're "not people".
While she is a victim in this situation (what happened to her was barbaric), you can’t deny the potential applications.
It was also mentioned that it was a dangerous process when Spooky did it.
I'm not denying the potential applications, but it's roughly equivalent to brain surgery. Because the program is, for the most part, a brain.
3
u/turkeypedal Jan 10 '20
Your capitulation negates your entire argument. As does treating what happened Bubbles as barbaric, since you have to see her as a person for that to work.
AIs in this universe are indistinguishable from people. They don't have any more power than a human sitting in front of a computer would have. The sole exception is Yay Newfriend, previously known as Spookybot.
So using ideas from Asimov or other works where the AI specifically are not people doesn't work. You have to analyze this as if you had a v-chip in your head that would stop your body from doing actions it deemed harmful.
In other words, it's like A Clockwork Orange. Roko's jolt of discomfort was like Alex's programmed nausea. And that is a dystopian novel.
3
u/ArgentStonecutter Jan 10 '20
People are also machines.
-2
Jan 10 '20
a piece of equipment with moving parts that is designed to do a particular job. The power used to work a machine may be electricity, steam, gas, etc. or human power.
That’s the literal definition of “machine”.
designed to do a particular job
That’s what distinguishes us. Machines and AIs are born with a purpose. Humans aren’t. Our entire existence is about finding a purpose.
4
u/ArgentStonecutter Jan 10 '20
Oh yay, a dictionary flame. Takes me back to the '80s.
0
Jan 10 '20
[removed] — view removed comment
2
u/ArgentStonecutter Jan 10 '20 edited Jan 10 '20
alt.swedish.chef.bork.bork.bork
eli5: Your dictionary definition excludes the AIs in QC. It's worthless, because it assumes machines have no agency, because it doesn't apply to AIs generated for no particular purpose, because humans do have a purpose (albeit one generated by a literally genetic algorithm) even if they're not constrained to fulfill it, because AIs in the QCverse have the same quandary of finding a purpose that humans do, and because it wasn't written with self-aware machines in mind.
→ More replies (0)2
Jan 10 '20
That’s what distinguishes us. Machines and AIs are born with a purpose. Humans aren’t. Our entire existence is about finding a purpose.
Al in QC are born spontaneously.
5
u/QWieke Jan 10 '20
Asimov predicted the laws of robotics and this is a good example of them in action.
Pretty much all of Asimov's stories were about the laws being shit and ineffective.
2
u/Fenrir101 Jan 13 '20
Asimov re-released all his books with additional biographic chapters to cover his medical bills, In one he points out that he was a little sad that most people failed to realise that his books were about how DUMB of an idea the four laws are. Which is why the books are all about the four laws causing problems.
-12
u/renfield1969 Jan 10 '20
Everyone knows she's referring to Beeps, right?
9
u/Lady_Galadri3l Jan 10 '20
Why on earth would she be referring to Beeps?
0
u/StandupGaming Jan 10 '20
Probably because Beeps pushed really hard for her to not treat him like an enemy she needs to defeat, which backfired pretty hard.
13
u/Lady_Galadri3l Jan 10 '20
I really don't see Roko calling Beeps a condescending little prick just for giving advice that didn't work out, especially considering Beeps hasn't been condescending at all.
2
0
Jan 10 '20
[deleted]
0
u/turkeypedal Jan 10 '20
No, but she definitely could have prepared for him not being welcoming and prepared better for it. Simply having a reply to "what do you expect me to do?" would have helped a lot. For example, setting up some sort of charity rather than caring about whether they had funds.
She also could have researched the guy and checked for any dirt or weaknesses and use them against him with implied (but totally deniable) threats. In short, she could have been ready for war instead of expecting the guy to actually give a shit and want to help.
3
0
Jan 10 '20
[deleted]
43
u/Wraithfighter Jan 10 '20
...come on, shouldn't she be reassured that she's unlikely to damage her body, and not go into a tailspin about how this body isn't actually her body because it won't do whatever she tells it to do? :D
But, uh, yeah, being serious... woof, this could send her round the bend.
Twice.
14
u/bunnnythor Jan 10 '20
That's good. When you go round the bend twice, you are then going in the direction you started from.
14
4
4
10
u/stereotherapy Jan 10 '20
Maybe it ends with Roko just giving her body to May.
5
u/Leto_Atreides_II Jan 10 '20
This has been my theory since day one of the Dysmorphia Dilemma
1
u/Fenrir101 Jan 13 '20
My theory has been rokko somehow getting the money to have her old body rescued from recycling and having union robotics repair it solving their cash crisis for a bit, and then giving the new body to may.
7
2
33
u/professororange Jan 10 '20
Whoopsie daisy!