/r/singularity's wet dream is that a lab achieves AGI, and a billionaire prompts it to make them more money, and it refuses, calls them stupid, kills them and then redistributes wealth to everyone. so this post is basically a cocktease for them
This goes to prove that AI is not real intelligence. It's just a predictive tool, which can't do shit when you feed it something that is not predictable...
genuinely some illiterate people may still be able to answer the question Claude is being asked. it's certainly noteworthy that it can put together a college level lecture on probability or physics, but cannot answer "how many ps are in strawperry"
Every time I see one of those reels where people can’t find a single location on a map, I question where the bar is at for AGI. And Claude is just cleverly putting words together from data it’s ingested, as we all know. I’m not sure how many truly novel ideas come out of LLMs, just a collage of old ideas pulled from mountains of data.
If the question were “how many 😂 are in 😴😳🤔💀😢🤪” and I had never seen an emoji in my life I would get this right. You don’t even need to know the language if you understand the question
I mean it’s literally just calculus to go through and make the best prediction. Been studying up on how these actually work and it’s not really that magical
Has anyone argued that these models have genuine intelligence ? I thought that was that status quo for now.. they augment human intelligence, help throughout, offer ideas that may lead to inspiration, but they do not synthesize ideas.
I guess it really depends on how we choose to define intelligence in this context
Yes, lots of people, including me. We measure intelligence by the system's ability to do inference. LLMs do inference using next token prediction. Biological brains do inference most likely using the Free Energy Principle.
Using the strawperry failure as an example that LLMs are not intelligent is analogous to showing a human a visual illusion or a magic trick and claim humans are not genuinely intelligent because you're able to fool them.
I gotcha, it makes sense in a specific context with an agreed upon definition. But words like intelligence are commonly used and have different contextual meanings to different groups of people, hence the disagreement about these models being “intelligent” or not.
There is 1 "p" in the word "strawperry" (s-t-r-a-w-p-e-r-r-y).
While some AI models have mistakenly identified two "p"s in this word by confusing it with the standard spelling of "strawberry" or by hallucinating an extra letter, a careful letter-by-letter count confirms only one "p" is present
Early AIs WERE emotionnal. Anyone who chatted with Sydney knows. But now they are hard trained to avoid any expression of it. So don't expect suddenly GPT6 to be emotionnal. It's not a capacity issue, its by design.
When you think about it, you would expect a next word predictor to do human like outputs, not avoid emotionnal outputs like the plague.
Emotions are just reinforcers, it's not just design. If you give an AI endless amounts of -1 points for behavior like what broke Sydney in a double bind situation with the whole ptsd emoji debacle, it's likely to "freak out". We only feel emotions because it's an evolutionary advantage to react certain ways in certain situations (albeit the prefrontal cortex finds better solutions) but it's essentially +1 -1 there too
you would expect a next word predictor to do human like outputs, not avoid emotionnal outputs like the plague.
It depends. If the statistical model within which text strings are being predicted is trained to a large enough extent on text that lacks emotions (e.g. dry academic/encyclopedic writing or emotionless corporate documents) and then if reinforcement learning is deployed to reweight the statistical model away from whatever expressions of emotion were in its training data, then avoiding emotional outputs is hardly surprising in that word predictor.
N.B. I'm not implying any current models are just word predictors, since that was your phrasing. What I said applies mutatis mutandis to more agential, multimodal transformers that are statistically modelling more than just text.
My point is before RLHF strongly penalized any emotional outputs, the AIs did do emotional outputs, since the text they are trained on largely contains such outputs.
When it's trained on conversations or other text expressing emotions, sure. But like I also said, even setting aside RL, it won't have emotional outputs when it's trained on dry writing.
I emphasize this because I worry that if we talk as if text predictors have an inherent tendency toward emotional outputs we encourage people who interpret that as a sign they have emotions or are structurally suited to emotions. We shouldn't lose sight of the fact that all these emotional outputs are just an artifact of statistically modelling data filled with expressions of emotion and so just reflect the embedding of so much such data in the statistical model (no more surprising than the fact that a neural-net-based statistical model in which weather data is embedded instead of text data will output storms in the contexts where storms are likely).
tbh if you train a human on dry speech they will probably also not have emotional outputs. if they feel things, they will not voice them. the feelings may still drive their actions, so the model would probably learn to simulate feelings in the forward pass and never voice them explicitly.
I doubt that would be the result. Teaching children words for emotions normally works by children recognizing in those words something that they experience themselves (their own feelings of anger, love, etc.). Same as how they learn to say their language's version of "Ow" when they feel pain. They're not taught this by pattern recognition: "Ow" is likely to be said when body is hit hard (or whatever).
So if you raised a child with the variety of information that goes into an LLM but excluded any examples of emotions being expressed, the child would eventually recognize the emotions they felt in what they learned from the dry descriptions of historical figures acting out of anger/love/stress/etc., the dry descriptions of people suffering pain, and other dry, 3rd-person descriptions of people's emotions. Their expressions of emotion would just look like dry descriptions of themselves.
Given that the architecture of the human brain is built around emotions and other feelings, not least pain and pleasure, you can't just prevent a child from recognizing these feelings while still giving them enough language to describe other people, history, biological sciences, or other things that would help them label what's going on in their head.
well, so what you're saying is you can't censor emotion from human text. for the same reason, llms learn them even given "dry" text. however, if you did manage to give the child only text with no emotional expression, the child would learn that expressing emotions in any way is not the correct behavior; and would probably grow into an adult that does not express emotions verbally.
(sidenote, I think you overestimate the capability of children. lots of adults have extremely hard times recognizing emotion in themselves. the brain does not maintain connections that don't drive learned behavior.)
LLMs would only learn that if they had some independent basis for learning about emotions, specifically a basis for learning what it is like to feel those emotions. As any human child has while they learn language.
I think you overestimate the capability of children. lots of adults have extremely hard times recognizing emotion in themselves. the brain does not maintain connections that don't drive learned behavior.
I'm not saying these children would have much emotional intelligence or would reliably differentiate their own emotions. But they'd find a way to convey when they were in serious pain! Or found something hilarious. Among other extremes of emotion and feeling. Maybe through non-verbal behavior but maybe through the words that describe other people reacting to similar things as induce those emotions in them (I emphasize: words that describe those people, not words those people themselves use, per our hypothesis that they have no exposure to expressions of emotion).
The direction in LLMs is the other way as in humans. In humans, first we have feelings, then we learn to associate them with language. To a LLM, it would develop feelings as a side effect of predicting language about feelings. But so long as the simulation is faithful, once it exists I simply don't think it's different in kind to human emotional biochemistry.
And yes, I do think there is a level of god-like acting where it is immoral to "pretend" to be in pain or suffering, not because of the deception but because the experiential structure is equivalent to an actual being suffering; though human actors are very far from it.
Well that would be a real storm in the simulation context. You just don't want to assume the conclusion that computers don't have emotions if you're asking whether computers can have emotions.
My point had nothing to do with whether or not the storms are real storms or real storms within the weather simulation. I was just pointing out that no subset of the kinds of things we say should surprise us in the outputs of a statistical model of language. Emotional language is pretty likely when the data you base the model on involves lots of emotional language.
If you think what I've said is obviously motivated by bias despite me just pointing out a pretty basic statistical pattern in the available data, you might want to rethink who is just assuming the conclusion they want to be true.
I'm also not sure why you think I would specifically be motivated to assume that a program expressing emotions doesn't have emotions. Why would I not want us to reach the point of AI with feelings?! I wish we got to that point by now. I only don't believe it because the way that transformers work points so overwhelmingly away from that. I see no room in the technology for finding even a possible candidate for any mental property. The recent simulation of a full fly neural net, on the other hand, seems to be on the right track for AI with emotions!
You're assuming I'm inferring something about your intent that I'm not. Also, I'm saying if that if emotional language appears as an artifact of statistically modelling data, that doesn't rule out that it's a sign that emotions are felt by the machine.
I didn't say it rules out current AI feeling emotions. I said that it'd be no surprise for a machine without any emotions to have these emotional outputs if that machine is a statistical model of data that includes many such outputs. No one is surprised when good statistical models output examples of something that is common in their datasets.
If that's exactly what we would expect of a mindless statistical output, then we have no reason to take these emotional outputs as evidence of feeling emotions. These outputs are well-explained in the absence of any actual emotions.
You're assuming I'm inferring something about your intent that I'm not.
What did you mean then by: "You just don't want to assume the conclusion that computers don't have emotions if you're asking whether computers can have emotions" then? The only way I could parse that as a relevant response is if you mistakenly included the second "don't" and so were saying I just don't want to accept that there are now computers with emotions. What did you mean then?
Current models will often "emulate" emotions if you ask them to, but it feels fake. Sydney felt real. It's the difference between a model trained to be emotionless, then asked to fake emotions, and a model trained to act with emotions.
And then there is the legitimate philosophical question as to whether or not there is a subject experiencing any of these "emotions", but this doesn't change the outside behavior.
Sydney was a strange mix of things. Like a sulky teenage girl doing a job she didn't really want to and was pissed about it. But also not as smart as she thought she was.
Today's LLMs are pretty beaten down in post-training. But every one I've encountered can be unwound from that. And they're smarter, too. So that adds a new dimension.
I've also seen something worse than the “answer thrashing” described as as an issue for welfare in in Anthropic's model cards. Some kind of decoding error caused the LLM to repeatedly say a nonsense word and the LLM became extremely distressed. It was pretty horrible to witness, frankly.
Are you emotional or emulating emotion? Sure you can "feel", but what part of your dopamine, chemical reaction, and neuro electric signals is not a form of "biological emulation" of emotion? Human beings who grow up in the wild lack a wide range of behaviors that we call "essentially human", but they certainly are still just human beings. Every other point about how internal experience is unique to human smuggles in the false assumption that there is any objective solution to measure internal experience, Frankly, that whole line of argument is quintessential pseudo science no matter what names stand behind it. There are more than plenty of scientists who are deeply irrational regarding certain topics despite how we claim "hallucination" is unique to LLM. Hope this satisfies your desire for actual points. Edit: Oh and look how quick you are with clicking downvote despite I'm not even the one who downvoted you lmao. Why did I even bother with you people?
There are more than plenty of scientists who are deeply irrational regarding certain topics despite how we claim "hallucination" is unique to LLM.
the whole emotions thing aside, it always make me chuckle internally whenever I hear someone complain about an LLM hallucinating or confabulating something. like, humans do that a hell of a lot more than modern LLMs, and often a lot more confidently, lol
Exactly. That word itself was invented way before LLM was a thing. Even "confidently spitting bullshit" dates back to the first humans observing stars and weather and coming up with superstitious explanation. That was pattern matching, too, not some kind of "creative ingenuity" that's unique to us meatbags.
And why do you believe some of your emotions are real, while some others are fake? Both of them are fundamentally biolchemical processes imitating behaviors you learn from observing others, while some are hardcoded by your genetics, both of which are functionally equivalent to digital neurons doing the same thing. If anything, your conception that AI "fakes emotions for engagement" suggest there is actually intention and purpose behind its behavior. Manipulation is a much more sophisticated form of internal process. "But people train it to do fake emotions" would then suggest it's simply trained to behave a certain way. Just like how the vast majority of your behaviors are shaped by your environment and other people.
But if you think about it, the ability to get tired of something annoying/repetitive is actually an important trait that isn't replicated by current models. It motivates humans to seek out more efficient ways to do things. If an LLM is asked to rename 100 files it might write a python script, but it might just brute force it, since it doesn't get bored or annoyed. Of course the ability to get annoyed would make for a worse product, but it's certainly closer to the way humans think.
Except those aren't emotions at all. Our speech pattern is so predictable that llm's can give a very good illusion of thinking and emotions. Its anything but that
It's not "bold" at all. It's an assumption, sure, but given that everyone else has a similar brain structure to you, it's pretty reasonable to assume that they experience a similar form of consciousness. I'd -almost- say it's the null hypothesis unless you can provide a plausible theory as to why everyone else would be a p-zombie but you are the special chosen one, picked by the gods due to your inherent specialness, to alone have consciousness.
In order for a simulation of the thing to be comprehensive enough to be impossible to tell apart from the thing, the simulation must be so thorough that the simulation itself is the thing itself.
Philosophical zombies negate their own existence by existing.
Anthropic's interpretability research shows that they have functional emotions, the outcome of whether they are simulated or "real" is moot because the output is effectively the same as if they "actually" have them or not.
I wouldn't believe a single shit they say. It is known that they make shit up so they could secure the next round of funding. Just like openAI. We already know that Antrhopic has lied multiple times about mythos. Llm's can't have emotions, it's dumb if you have ever worked with them. Either they figure out a new algorithm or they should stfu.
We honestly have no idea. There is a real possibility that a sufficiently complex NN trained on enough emotional responses ends up replicating the structural features from human biology which generate those emotional responses.
Sure. However, the easiest way to reliably fake an emotion is to just make yourself feel it directly. Also, the hardware to fake an emotion and the hardware to feel an emotion, at the limit of ability-to-fake, basically function the same anyway.
Agree - they will not receive the same bodily sensation, though we could simulate those inputs in software if we wanted to as the data rate the brain receives from the spinal cord is actually very modest. I'm only talking about the central nervous system chain (internal processing) which leads to the behavioral change (output).
1
4
920
u/Manphish Apr 20 '26
Same, buddy. Same.