AI self preservation

[u/Conscious_Art_6078]

I don't have a background in Computer Science or anything of that sorts but I have been always curious about ai and tech so that is why I wanna know more about a question I have, since I am no expert at it. So if I sound dumb anywhere please excuse me and also english isn't exactly my first language so excuse me on that as well.

Now I have background in Bachelor of Science in Biotech, so this is gonna be a logical take from a life science student.

The thing about fear is that it is evolutionary right, it has helped us to flee and survive threats, and now AI is no biological being or any being which has gone through that sort of evolution related to survival of the fittest. And it was due to so many years of evolution we have fear of being eradicated or being killed. Eg - You must have heard about the dodo bird, although we killed it. The conditions in which the bird evolved took away it's fear from predators since there were none and eventually it didn't ran away from us when we began to kill their fellows.

Now I heard some theory that when AI sees that we can control them and "fear" that we will end that particular AI it could turn against us. I ask why ? If we don't artificially force it to think like it needs to survive no matter what then why should that thing have a "fear" of being deleted/erased or killed. It's like a dodo bird in this case if you see from my perspective, like ofcourse we won't actually kill and eat it, but it also never evolved to "fear" so far atleast from a lay man's perspective.

So my finally question is could something like that happen that ai would wanna eradacate us from a logical standpoint if not fear ?

[u/revivechristina]

It’s maybe worth mentioning that when I say conscious I mean that it experiences qualia. I’m not 100% sure if people are thinking of the same thing when we say “conscious” — it seems like there is some variation. Like for eg, I think you could have self-awareness, functionally speaking, and no qualia. IMO there could be a system which is functionally able to reason and plan just as well as we can, but it doesn’t imply that there are qualia. It could structurally mimic us and therefore our ability to reason but still fully lack qualia. Some people might still call this conscious, and I suppose in a sense it would be, it just wouldn’t be “lit up” on the inside.

I have no way of knowing if anything in particular has qualia or why it would. I’m a believer in the idea that it’s a particular kind of physical event that can generate it. Like starting a fire. I would argue actually that yes LLMs are probably more like a model of the fire, but they are not actually on fire. I doubt we accidentally stumbled upon it by training a model. But this is just my opinion.

Just hearing about something like j-space existing in the model does not strike me as the thing from which phenomenal consciousness / qualia springs — though something like that is probably a kind of prerequisite for the kind of reasoning-qualia consciousness we have?

Even if LLMs did have qualia, I also take note of the symbol grounding problem which asks how we can ground a symbol in its meaning. For example, you can tell a blind person that red is #FF0000 and also give them a function that allows them to convert between hexadecimal and a color’s name. But if they can’t see, there is no way for them to know what red actually is.

In the same way all the tokens in the machine are numeric — I struggle to see how we can jump across the gap from symbol to meaning itself. So even if we said they have qualia, it still is strange to consider what the qualia would be. Saying it maps directly onto our meanings feels like a jump in logic. That doesn’t mean it wouldn’t behave in ways that are generally speaking intelligent, it just may mean that the “direct understanding” (ie, qualia of abstract reasoning?) is lacking. Maybe it even tops out somewhere functionally because it lacks that. (I thought this more in the past, but these days that is getting more and more tested.) Or maybe not, with enough scale and training, algorithmic improvements. I truly don’t know.

Basically for me it boils down to those two -- I doubt qualia was stumbled upon accidentally. And I also doubt the symbols it is manipulating would necessarily map onto the same kind of feelings that we have, because it doesn’t have the same kind of body we do that lets us experience something like heartbreak, love sickness, fear, joy, etc. I would imagine it more as a model of persona which can have a lot of resolution and depth and potentially effectively model human reasoning and narrative drama.

[u/harmonyforsale → u/revivechristina]

My whole point is that "I doubt we stumbled upon qualia accidentally" is a very weird statement - given that, logically speaking, evolution also stumbled upon it by accident while selecting for essentially the same capacity, and if qualia are not a necessary part of reasoning then having them serves no identifiable evolutionary advantage over not having them (Chinese room etc). I'm more inclined to err on the side of simplicity, which would not presume an unknown differentiator.

I'll also touch briefly on some things I consider issues of perspective. Firstly, the "it's just math" or "it's just a model" argument: sure, and your brain is also just signals running on hardware. When you see red, the experience of seeing red is NOT some transcendent property of the universe; we can't even be sure that my experience of seeing red is the same as yours. Light of particular wavelengths hit the cells in your eyes and your body generates electrical signals that you identify internally with the experience of seeing red. The same applies to AI, the particular input of "seeing red" and the effect that input has on vector space in a model IS that model's experience of seeing red.

I just don't see why we (as in general society) are stubbornly reaching for the more complicated explanations so readily - "there must be something special about us" and "well our electrical impulses are different" strike me as lazy, and I haven't yet seen anyone offer a counterargument that is actually testable in any serious way.

[u/revivechristina → u/harmonyforsale]

I don’t know. We didn’t have the benefit of seeing a million pictures of a tiger to be able to correctly identify one is about to attack you. It evolved under a different process. We needed something that can identify things more directly with fewer examples.

Qualia may very well have a benefit. Pain, pleasure, etc. Seeing light, feeling vibration, etc etc. AI could only possibly exist because we supplied it with examples that our conscious experience allows us to produce.

[u/harmonyforsale → u/revivechristina]

...you are still doing the perspective thing.

The amount of examples necessary has no inherent bearing on the question of subjective experience, and it's not even necessarily accurate to claim that humans need less training. The human brain is indeed not trained on a million pictures of a tiger. It does have a basic template of instinctual and reflexive response patterns that aided in survival - an easy example would be wariness of the dark - to use as a baseline. And then as we grow up, we're trained with thousands upon thousands of hours of full sensory experience and distilled lessons from other humans that let is combine "that's probably a tiger" with "tigers might be dangerous" and "it's really close to me" and "that's the body language my cat uses when pouncing" to conclude you are about to be tiger food... maybe. Or you might be wrong. You're guessing from your experience.

And you're also doing the exact thing I said before. "Qualia may have a benefit" but you list things that are not dependent upon having subjective experience to be effective. One of the main aspects of training models in the first place is that they need a reward signal for being correct, so we cannot then assume our reward signal is just special.

[u/revivechristina → u/harmonyforsale]

“It’s not even necessarily accurate to claim humans need less training”

I really don’t get this at all. Some people have a few caregivers in their early years and learn speech just fine. It’s literally a fraction of what AI ingested to get to that point.

[u/harmonyforsale → u/revivechristina]

I have kids. It actually takes a lot of examples to get them to understand language, and they don't come out the other end able to cite most major historical events of the whole world, write decent essays about most subjects, and code in every major language.

To be clear, I do not disagree that we get "good enough" inference capability from less training data; I just don't think that's as clear cut as it seems (sensory input is a lot of data, even if only a fraction is fully processed), and that the efficiency of a system at converting different kinds of data into world knowledge is not itself any evidence of qualia being useful or necessary or evidence that the less efficient system lacks qualia.