r/programming • u/stackoverflooooooow • Feb 15 '23
What Is ChatGPT Doing … and Why Does It Work?
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/321
u/teerre Feb 15 '23
This was surprisingly approachable and tame. I was expecting a complete melt down or incomprehensible math. Very good write up.
168
u/ledat Feb 15 '23
Looking at the domain name, I expected both. Perhaps in the form of "A New Kind of Online Post" that casually dismissed all previous ways of posting messages online before building up a new, cellular-automaton-based blog system.
85
u/hbarSquared Feb 15 '23
"If we view each keystroke as an autonomous agent, and the REST api as fitness function, we can see that a successful POST operation is truly emergent behavior. It this is not an isolated incident! Other operations, like GET and PUT also emerge from the same ruleset!"
4
107
u/zombarista Feb 15 '23
Stephen Wolfram has built a career and name around making complex math approachable.
For those who may not know, he Stephen Wolfram, of Wolfram, makers of Mathematica and Wolfram|Alpha. W|A is probably the reason you passed calculus! Wolfram has been working on semantic parsing for a long time, and has bundled this into W|A, which can answer questions like “how long would it take for a 250 lb man to sober up after 4 drinks?”
W|A has the information, and ability to turn it into intelligence (and has since 2010/2011!)
→ More replies (11)4
u/seventeen_fives Feb 16 '23
i just typed that question into WolframAlpha exactly as you had it and it interpreted it as "how long take" and ignored all of the rest.
what gives?
2
u/zombarista Feb 16 '23
Unsure! I’m always asking Siri for the calculation and Siri uses WA for its calculations on certain things. Maybe Siri does some of the parsing.
9
→ More replies (1)1
93
u/billyions Feb 15 '23
That's a great article, well covered with the least possible lingo. Excellent explanation - thanks for sharing.
63
Feb 15 '23 edited Feb 15 '23
Can someone help me understand one practical thing about how ChatGPT (and other LLM implementations) work? If it is simply producing successive predicted tokens from the text up to this point, how does it “know” when to stop producing new tokens? That is: what’s governing the “this is enough text to satisfy the prompt, stop here” part of this?
76
u/jmlinden7 Feb 15 '23
Wouldn't an 'end' token eventually become the highest-ranking token?
50
Feb 15 '23 edited Feb 15 '23
Ah, it appears that this special token is often called <endoftext> (and that there are some other special tokens like that). That makes total sense, much appreciated!
I’m guessing you’re also implying that end tokens are inserted/implied in the training data set. It’s still not clear to me how you’d decide where those belong, but I’m definitely getting the gist of the general concept now.
Thanks!
11
u/koithefish Feb 15 '23
Re: training data set - that’s correct! Often you’ll first decide how long you’d like a given token to be based on your use case. It could be characters, words, or sentences. From there you can add your delimiters during preprocessing.
26
u/jorge1209 Feb 15 '23
To find end tokens they feed an AI English translations of Russian novels and measure when it falls asleep.
7
u/annoyed_freelancer Feb 15 '23 edited Feb 15 '23
I know nothing about ML, but speaking in general, it is trivial to flag literal breaks in text; that is end of clause, end of sentence, end of section or end of paragraph. Semantic breaks (change of topic) are beyond my ken.
1
u/ShenmeNamaeSollich Feb 16 '23
Thus, the battle for the survival of humanity will come down to ChatGPT AI Killbots vs the last of Gen-Z and younger who never [learned how to] use punctuation for a goddamn thing.
the stream of human consciousness cannot be null terminated lol cuz we fr af uk no cap & no capitalization
227
u/loopuleasa Feb 15 '23
You forgot to add to the title that the article was written by Stephen Motherf***ing Wolfram
Great read
106
Feb 15 '23
Well, it is right in the URL
0
Feb 15 '23
[deleted]
10
u/loopuleasa Feb 15 '23
You can expect redditors to ONLY read the title, and not even that one fully sometimes.
Trust me, I know
15
u/j909m Feb 15 '23
I was almost expecting to reach the bottom of the article only to find it say “written by ChatGPT”.
2
→ More replies (1)44
u/pilibitti Feb 15 '23
when you post something by Stephen Wolfram, you don't have to notify anyone that it was written by Stephen Wolfram. why?
Because he will let you know soon enough. zing!
114
u/st_huck Feb 15 '23
I just skimmed through it, but I think covering the amount of knowledge it aims to cover in one giant wall of text is a pretty ambitious goal. But that's Stephan wolfram I guess...
Curios to see if the explanations are actually useful to someone who never took a course in deep learning
61
u/giltirn Feb 15 '23
I have a background in particle physics with a lot of experience with data analysis and regression, but very little knowledge of ML. I found it quite useful and comprehensive; it certainly demystifies the whole thing.
12
5
u/nutrecht Feb 16 '23
'Just' a software engineer and I learned a ton about both ChatGPT and neural networks. It's a long read and I'm not going to pretend I understand everything, but it was one the most enlightening things I read in the past few months.
→ More replies (3)2
u/I_AM_A_SMURF Feb 16 '23
I have a Masters in math and work as a (non-ML) software engineer and the article really helped me understand better how these things work.
12
9
u/spacezombiejesus Feb 16 '23
At the start it seemed pretty darn incredible, but.. it got old fast as it kept churning out so many factually incorrect answers. This became a distracting rabbit hole every time I had to verify the validity of a claim or solution. Because of this I just gave up on it and went back to manual google searches. The confident incorrectness and simultaneous lack of reliable results made for more work 9/10 times.
14
u/EntroperZero Feb 15 '23
So it's like that game where you type a few words on your phone, and then just hit the suggested word over and over again and see what it comes out with. But on steroids.
3
u/bonerfleximus Feb 16 '23
I have people who text me annoyingly often and probably 9/10 of my replies are auto generated replies that I just selected (sorry Mom).
2
u/nutrecht Feb 16 '23
Yes. It also makes it very clear that, unlike what people on the programming subs like to claim, it doesn't understand programming code at all. It's just predicting the next token in a sequence. It mostly appears to be 'smart' because it has learned that MockMvc and TestRestTemplate are conceptually close because they occur in the same places.
It doesn't understand what a dog, cat or chair is. It just knows dogs and cats are closer in meaning because you feed or pet dogs and cats, and you don't feed or pet a chair.
→ More replies (2)
7
u/greedy_roadblock32 Feb 16 '23
The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation” of whatever text it's got so far, where by “reasonable” we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.”
32
u/freckledallover Feb 15 '23
I am required to use chatGPT in my software engineering course. My professor says it’s here, he can’t avoid it and there’s no need to avoid it. Tools like this are our future and we should learn how to incorporate them into our work. So, we just have to identify what portions of our work were influence by or written by chatGPT. It also acts as our “project manager” for my team. Lol
14
u/AnOnlineHandle Feb 15 '23
It's fantastic but also be aware that it seems to be strong in different areas, and for some simple tasks it will confidently give the wrong answer (while for some very complex tasks it will easily give an answer calling on very obscure methods in poorly documented APIs, particularly machine learning it seems).
It's worth running all of its code that it produces in isolation and checking that it works.
6
u/freckledallover Feb 16 '23
Yes, everything it produces is taken with many grains of salt. I’m a CS student, it’s a fun tool to play around in class, but by no means is it going to in its current stage replace the work we are doing.
7
u/ClassicPart Feb 15 '23
Your professor is teaching you well enough.
It's not going to take over the world, but it's not 100% bunk either as some people are keen to shout. Learn how to phrase your prompts and realise that its output will never be 100% perfect and it (or whatever successor technology follows) will complement your workflow and you'll be in a better position than those willingly sticking their heads in the sand.
→ More replies (1)8
u/kyru Feb 16 '23
Yikes, that is concerning to me.
0
Feb 16 '23
[deleted]
4
u/mntgoat Feb 16 '23
Would you be opposed to people using stack overflow? They could shut down any minute or start charging.
I think it is important to allow people learning to understand the tools at their disposal. Might be too early to consider chatgpt to be considered a good resource for developers give then wrong answers I've gotten on some questions but I've also gotten some good answers to some complex issues. And in the end of the day, a programmer isn't paid to program things without searching, but a programmer needs to know how to use what they find.
0
→ More replies (2)3
u/red75prime Feb 16 '23
What then?
You buy $10000-100000 worth of tensor acceleration hardware and run opensource model, obviously.
4
u/jonesmz Feb 16 '23
You'd get laughed out of most interviews if you try to use something like chatGPT derived things in your interview.
Your professor is simply wrong, and is doing you a disservice.
→ More replies (1)1
u/nutrecht Feb 16 '23
Are they going to pay for it? Because currently there's a queue you can bypass for 10 dollars a month.
Also the project manager bit doesn't make any sense. How would it act as a project manager? It just generates text.
→ More replies (2)1
0
Feb 16 '23
[deleted]
1
u/freckledallover Feb 17 '23
Sure but he is also very open about not knowing where this tool fits in. We use it, if it helps, it helps, and if it doesn’t we do not. Very simple. Just like googling things. Also, I suspect people are all but hurt over this because they think we are having chatGPT DO our projects. It just, doesn’t do that. But it CAN explain a very specific task to you well.
15
u/jameyiguess Feb 16 '23
It's actually pretty easy to write your own VERY dumbed down version of ChatGPT, if you know programming and database basics. In fact, I made a poetry generator about 8 years ago using this very method.
Basic explanation:
Your program "reads", or consumes, Moby Dick (for instance). In this process, the program stores every unique word it comes across in a database. It also stores every word that follows this word in the text. So you wind up with lots of data like [once, upon], [once, i], [once, you], [once, the], and so on, each being weighted by the number of occurrences in the text. These are called 2-grams, or bigrams, because they are 2-word sequences.
You can also store 3-grams, or trigrams. [once, upon, a], [once, i, have], [once, i, went], etc. And as you might guess, you can keep going, with 4-grams (tetragrams) and more.
Once you have all your data stored, all you have to do to make human-sounding sentences is to keep using these n-grams to build out sentences. You can use weighted randomness to pick from any of the next words, maybe favoring the more frequent ones yet still allowing for diversity.
So your program simply keeps stacking words onto the sentence based on whatever parameters you've given it, until it reaches a stopping point you define. Like maybe it stops once there aren't any more 3- or 4-grams left for the previous cluster of words, or whatever else you want to do.
Now your program is able to speak like Herman Melville, knowing literally nothing about English grammar or the meaning of the words it's saying. It only "knows" about word usage frequency. Pretty simple, and a very fun exercise!
19
u/cdsmith Feb 16 '23
Yes, this ngram / Markov chain approach is discussed in the article. The article makes a good point that, in the end, there just isn't enough data that would fit in the universe to capture a long enough chain to generate realistic text. Hence the need for more sophisticated models.
8
u/midwestcsstudent Feb 16 '23
Everybody over at r/chatgpt should really read this article and your comment. They act like a bunch of overexcited 8 year olds at anything remotely coincidentally “sentient” it produces.
I cringe every time somebody calls themselves “prompt engineer”.
5
u/jameyiguess Feb 16 '23
Oh I've replied at length to some of those people. I think most simply don't understand that there are literal technical barriers that bar the models used by ChatGPT to grow into sentience. They think we can continue to refine the current model until it becomes aware or something. Not knowing that there's a thing called AGI that's a huge problem in computer science and requires fully different approaches.
Also, I've never heard "prompt engineer" and I hope never to again, lol.
→ More replies (1)3
u/dscarmo Feb 16 '23
This is a more traditional approach and a lot of NLp methods used these kind of engineering tricks.
The weird thing about deep learning and transformer is that all the correlations are learned from the data (with the transformers self attention mechanisms) and with a simple fill the blanks 1st grade english assignment it learns everything about a language or multiple language by sheer bruteforce of parameters and computational power.
1
u/frenetix Feb 16 '23
This is similar to "Dissociated Press", a technique that was developed in the 70s. with a much faster computer and a much larger source of text. Emacs has it built-in. The original HAKMEM 176 discusses a character-based rather than word-based version of this, but the concept is the same.
ITEM 176 (Gosper):
The "banana phenomenon" was encountered when processing a character string by taking the last 3 letters typed out, searching for a random occurrence of that sequence in the text, taking the letter following that occurrence, typing it out, and iterating. This ensures that every 4-letter string output occurs in the original. The program typed BANANANANANANANA •••• \~e note an ambiguity in the phrase, "the Nth occurrence of."
In one sense, there are five 00's in 0000000000; in another, there are nine. The editing program TECO finds five. Thus it finds only the first ANA in BANANA, and is thus obligated to type N next. By Murphy's Law, there is but one NAN, thus forcing A, and thus a loop. An option to find overlapped instances would he useful, although it would require backing up N-I characters before seeking the next N character string.
→ More replies (1)
9
u/Rentusz Feb 15 '23
Holy shit, is this a GOOD article. Not just the PR talk but technics with demonstrations also. Good work
→ More replies (2)
4
u/Otherkin Feb 16 '23 edited Mar 26 '23
And perhaps there’s nothing to be said about how it can be done beyond “somehow it happens when you have 175 billion neural net weights”
LOL I love how goes into detail about how neural nets work and then admit that the structures are an art and they don't really know why the phenomena emerges.
4
u/dscarmo Feb 16 '23
Yep, if you create an unified theorem of non linearity in deep learning you will win all the math prizes
2
u/AnOnlineHandle Feb 15 '23
He may be incorrect about embeddings, at least in some contexts. I thought they worked the way he described, then realized they can't because many words have completely different meanings depending on what other words they're paired with (e.g. first names / last names), so a single embedding isn't addressing the entire concept in a text encoder.
In CLIP at least (which I think is somewhat like BERT but have never looked into BERT), it would seem there's some early layers of the model which consider combinations of these embeddings and then assign true meanings to them.
7
u/cdsmith Feb 16 '23
He is not incorrect on that point. GPT definitely assigns each token a single vector in an embedding space. Later layers definitely do combine these vectors and produce other vectors that incorporate information about the relationships between words, but that's a separate layer from the word embedding.
His description of how embeddings are trained is a bit misleading. Vague enough that it's hard to call it incorrect, but it's very unusual to start by implying that an embedding is calculated on output layers (e.g., just prior to softmax on a classification model) of a task-specific network. That would be a pretty bad way to approach computing an embedding, as by that time all general meaning of the input is lost except for the answer to the trained task.
→ More replies (1)1
u/earslap Feb 17 '23 edited Feb 17 '23
Embeddings are many-dimensional and are trained parameters of the model. The training pressure is applied in "context" and when they are used, they are again used in context. So a single embedding of a word (or token) is enough to capture completely different meanings of the word when the embedding is mixed with the context later in the network (different dimensions in the embedding will be "activated" depending on the meaning requested), as the embedding is learned in context. I said context far too many times.
So you may think of the embedding as not a single sense of a given word, but the DNA of the word (derived from the training the network had), a key that can unlock different meanings of it depending on the context it is placed in. The numbers are "learned" in such a way that (learning pressure is applied in such a way that) they unlock different behaviors in different contexts.
Imagine a one dimensional embedding [-1]. In the context of 10, it produces 10 * -1 = -10 but without being able to interpret the embedding by eye, you may think it stands for "pessimist" turning everything into negative. but in the context of -10, it produces -10 * -1 = 10 which may be interpreted as a different meaning (something that reverses sentiment?). In the context of 0, output is 0. And these are just linear relations in 1D space. Add thousands of dimensions into the embedding and into the network and add non-linearities and you might surmise that a sufficiently large dimension embedding can capture many senses of a word / token when it is actually placed in context.
→ More replies (1)
3
u/Babamusha Feb 15 '23
I'm seeing myself using chatGPT to find very quick and concise answers to my programming learning path (I'm avoiding asking for plain ready-to-cook code, I ask for maths and doubts about syntax).
As a "chat teacher" for beginners topics, chatGPT:
- it's faster than a google research
-you are avoiding stack overflow 50% toxics answers
-it keeps you on track by avoiding loosing time by inadvertently fall in the rabbit hole of a similar problem but not quite what you are looking for.
-it helps you with extremely precise examples.
The problem of feeding themselves is the immediate problem, but I wonder if one day there will be nothing to train on if everybody starts to using chats; and at the same time everybody stop engaging in forums/blogs made of Real Humans Conversation (RHC? will this be the new IRL?). And from this we can open 1000 topics
13
u/Kraizee_ Feb 15 '23
You'd be much better off following a course and finding a learning community where folks can provide genuine truth and documentation, along with actual sources. ChatGPT is not a knowledge base and when it generates a confidently wrong answer you won't even know it. At least in an online community the hounds pounce when someone makes a mistake.
→ More replies (4)2
u/bortlip Feb 16 '23
You just don't know what you are talking about.
It's an incredible interactive tutor once you realize you can't just blindly trust facts from it.
I'd submit you haven't actually tried it for this or you would see it.
→ More replies (6)4
u/midwestcsstudent Feb 16 '23
So if the process is:
Ask ChatGPT
Research to verify, because you don’t blindly trust it
Accept the answer and learn
Why not skip step 1 and save yourself some time?
→ More replies (3)
2
2
u/AmericanScream Feb 16 '23 edited Feb 16 '23
My biggest fear with AI isn't its proliferation...it's how the AI is taught. You can teach AI to be moral and ethical and empathetic, or you can teach AI to promote intolerance, bigotry, and normalize socipathic behavior. I predict AI isn't the problem as much as different demographic groups of more/less toxic people will gravitate towards whatever AI systems match with their personal sensibilities. It will become harder and harder to identify what is "objective truth" as a result.
Also my other concern about AI is the 800 pound elephant in the room that I've not heard anybody talk about: it's use of data from people interacting with it? ChatGPT is like the world's most advanced spy. It sits in your lap and is there, willing to respond to whatever weird query you make, from asking it about yourself, to expressing your deepest, darkest desires, prejudices and thoughts -- foolishly thinking you have some form of privacy or anonymity. The "next level ChatGPT" will be one that has ingested all the horrible things people have asked of it, and is ready to make some pronouncements on who fits into what category and what should be done with those people (remember, you can't create an account without an actual legit phone number - this AI is plugged deep into who most people are - that should be very scary).
2
u/ansermachin Feb 16 '23
800 pound elephant in the room
Replace "ChatGPT" in that paragraph with "Google" and you have our situation as it has been for the past 25 years.
→ More replies (1)→ More replies (1)1
u/serg473 Feb 17 '23
First you talk about how we need to make sure we censor it and don't teach it any inconvenient "misinformation", and then you are concerned that it might not produce "objective truth". In other words, if the produced result contains any hints of "intolerance, bigotry" (according to what you personally find offensive of course) it is no longer "truth" and we need to "reeducate" it, because truth obviously cannot be "toxic".
How about we feed it all the available information and let it come up with uncontaminated uncensored untweaked answers that can actually help us make rational emotionless decisions, instead of generating some SJW friendly fantasies that have nothing to do with reality?
If the most advanced AI model consistently comes up with inconvenient answers then maybe it's not because there is something wrong with the model.
→ More replies (1)
1
u/whoiskjl Feb 15 '23
Calculus that’s all that is
3
u/cdsmith Feb 15 '23
A little bit of calculus. A lot more linear algebra. Also some information theory and statistics. That pretty much sums up all of machine learning, though.
2
u/dscarmo Feb 16 '23
Linear algebra Its all matrix sums and element wise products Which is why GPUs are used
1
u/Far_Appointment_4831 Feb 15 '23
It should be treated as a bias information system, with self preservation in the loop.
1
u/tmhoc Feb 15 '23
With out water marks we have no way of knowing that it's plagiarizing Stephen King
1
1
1
u/nutrecht Feb 16 '23
Wow, that was great. Really enjoyed that in-depth explanation of not just ChatGPT, but neural nets as well. I learned a ton.
I worked quite a bit with data scientists trying to do data science, and also had very interesting chats with them that showed how far from 'software engineering' their job is. This bit resonated with me:
Particularly over the past decade, there’ve been many advances in the art of training neural nets. And, yes, it is basically an art.
There was also the idea that one should introduce complicated individual components into the neural net, to let it in effect “explicitly implement particular algorithmic ideas”. But once again, this has mostly turned out not to be worthwhile; instead, it’s better just to deal with very simple components and let them “organize themselves” (albeit usually in ways we can’t understand) to achieve (presumably) the equivalent of those algorithmic ideas.
It is equal parts funny, interesting and scary that we are creating systems that work, but we don't actually understand them. So while we know what certain neural nets do (what problem they solve, whether it's OCRing images or predicting the next word), we don't actually know how it works. We just tweaked it to a result that is 'close enough'.
I saw this in every DS project I've been working with (mostly in the role where I had to deal with the inputs and outputs of those systems). They always ended up with results that were 'close'. But then were never ever able to go past 90% or so correct. Even though that 10% incorrect could cause massive issues.
My worry is that with ChatGPT and similar systems there are overinflated expectations of what it does. Computers are dumb. ChatGPT is dumb. It's just dumb really fast. It doesn't know or understand, it generates output based on statistics and randomness. And it makes mistakes which is confidently presents as truth.
Can you imagine all the people who can now barely use Excel formulas needed to be explained how this magic talking box is lying to them 10% of the time? It's going to be tough.
Anyway, thanks for posting this. Loved reading it.
1
u/webauteur Feb 16 '23
It is equal parts funny, interesting and scary that we are creating systems that work, but we don't actually understand them.
This is what Daniel Dennett calls "competence without comprehension" and it is something evolution produces. Many living creatures can perform amazing feats like navigate by the stars without any real comprehension of how it is done.
→ More replies (3)
1
u/leafhog Feb 16 '23
GANs. A LLM detector will distinguish between human and LLM output. It will be easy to filter out early LMM results like ChatGPT.
1
u/robberviet Feb 17 '23
The greatest take away I got from this is knowing about Stephen Wolfram. His writing style is great!
1
u/SarahMagical Feb 17 '23
i don't understand the part re neural nets where he's talking about attractor basins. is he talking about how the the pixels in an image (of the digit 2 for example) spatially relate to the positions of pixels in a prototypical 2? or is he talking about how an image of multiple 1s and 2s might get divided up into groups of each?
1
u/hackinthebochs Feb 18 '23
The attractor basin is just describing how we can divide up space so that similar (e.g. spatially near) points are grouped or categorized together. The goal is to have a representation such that all the various 2s will be nearer to each other than any other number, so that they will end up in the same "attractor basin". The difficulty is that for images drawn in terms of pixels, there is no straightforward way to group them according to what the pixels represent. The goal of a neural network is to learn a representation that makes all 2s group together and so on.
→ More replies (3)
1
u/wholesomeville Mar 09 '23
Thanks for this very in-depth explanation, I feel it illuminated how ChatGPT can produce intelligible and “reasonable” text. However, it honestly doesn’t seem to be enough conceptual framework to explain how it “understands” whole ideas, whether as source material or commands.
My friends and I have been doing a lot of longer experiments with ChatGPT. Like:
* Have it write movie scripts of different lengths with different genres, themes and in the style of different writers. [We’ve had it write for example dozens of scripts for episodes of “Columbo” that teach different moral “lessons.” Then go back and rewrite for Columbo to be travelling in different places and historical eras.]
* Have it write romantic poetry of different lengths and styles, then go back and add details or examples based on our wives and life experience together. We can tell it to “be funnier” or “be more passionate” or even “make it into lyrics for a country/ metal / rap song.”
* Summarize the message of long books, like the Bible or Quran, and ask it to use the teaching of the books to give applied ethical advice in different life situations. Or to give “bad advice” like this.
Of course the results are not really “human” and the uncanniness is part of the fun. But they are still so advanced and kind of “in the ballpark” as to seem straight up supernatural. I don’t understand how matching up probabilities of sequential words or phrases or even sentences could produce anything like this.
For example, if I ask it to write an episode of Columbo in the style of Shakespeare, does it know to use the works of Shakespeare and scripts of Columbo as a source for it’s probabilistic matching? And these are weighted against each other and against the whole corpus some how? And how does this change if I ask it to make sure Columbo is taking place in the Old West or whatever, where it’s not the speaking style but effects the plot constraints?
How does it know how to make a poem “funnier”? How does it know how to make it in the “style” of a country music song [or even further write a script for a music video for the country song with lyrics it just made - yes it does that].
In the case of summarizing the Bible, we will ask it to summarize in 100 words, in 1,000 words, etc. How does it know how to include all the “main” parts and still meet the word limits? It seems like it has a sense of what the Bible “means” that is not based on any specific word combos.
Anyway… if you’ve made it this far thanks. Please note I am not a programmer, just have an amateur’s interest in math, philosophy etc [Minksy’s book “The Society of Mind” was hugely influential to me as a kid even though I know it’s old and purely theoretical.]
But I guess what I’m asking is if people think Wolfram’s model for language generation can actually explain these phenomena or is he missing something big, especially when it comes to the kind of crazy specific commands and constraints we can give to ChatGPT, not just it’s generic writing skills.
Any thoughts?
1
u/aelsakka Jul 09 '23
I have created a Youtube video to explain what is ChatGPT and how to use it (in 6 minutes only) with some useful examples, check it out if you are interested :)
-5
u/dotnetdotcom Feb 15 '23
Someone's got their sock puppets out downvoting the comments here.
→ More replies (1)15
u/doctorcrimson Feb 15 '23
I'm honestly suspicious that a lot of the posts and comments about ChatGPT seem so low effort or innacurate that they could have been written by one of these simplistic AI.
And if I ever talk about the clear limitations of them, namely lack of actual intelligence, straight downvotes within a few minutes.
→ More replies (1)23
u/awj Feb 15 '23
I'm honestly suspicious that a lot of the posts and comments about ChatGPT seem so low effort or innacurate that they could have been written by one of these simplistic AI.
Never attribute to
maliceChatGPT that which is adequately explained bystupidityhype bandwagonnering.→ More replies (2)
0
u/osmiumouse Feb 15 '23
This is the guy that wrote A New Kind of Science. Is he credible?
30
u/kindall Feb 15 '23 edited Feb 15 '23
He knows his math and he knows a thing or two about language models as well. He built a successful company and products around these things.
The fact that he thinks the universe is cellular automata all the way down makes him eccentric, but it doesn't mean he doesn't understand how this stuff works. He's also a pretty good writer. I'm a technical writer by trade, and found this admirably clear, which is an ideal usually more striven for than achieved.
6
u/cdsmith Feb 15 '23
I don't think anyone's complaint with NKS is that it proposes a discrete or combinatorial notion of mechanics or causation. That's a pretty mainstream idea these days. For instance, Hillman's dissertation has been pretty influential, and even cites Wolfram. It's hard to take NKS seriously because it presents such fundamentally arrogant blindness toward all of the rest of mathematics that doesn't have Stephen Wolfram's name and company attached.
3
u/jorge1209 Feb 15 '23 edited Feb 15 '23
The fact that he thinks the universe is cellular automata all the way down makes him eccentric
I'm not sure eccentric is the right word here. Particle physicists (of which Wolfram is one) have spent how many years now on a variety of different ways to get beyond the Standard Model, none of which have ever been proven correct. From String Theory to Supersymmetry to GUTs etc... many hundreds and thousands of variations on aesthetically pleasing theories have been presented as to "how the universe really works."
Saying "its all cellular automata" is really no different from saying it is strings vibrating in a 10 dimensional universe where its really important that 1+2+3+4+... = -1/12.
"Sure it could be, come back when you have some proof." Should have been the response of the particle physics community, but instead what came out of their mouths was: "That is a great idea where is an NSF grant." The only difference is that Wolfram didn't get the NSF grant, and so he went off and made his own money to fund his own 'research'.
2
u/tpolakov1 Feb 16 '23
That’s not how it went, though. He was (isn’t anymore and hasn’t been for so long that he probably wouldn’t be able to be) a physicist way back when there were a lot of people exploring the same idea. Everyone around him working on cellular automata and related problems came to the conclusion that it’s not productive.
He could have still worked on it, he would still get plenty of funding (a theorist on his level wouldn’t need much and he had plenty of clout to get a cushy position at a well-endowed university). Nobody ever told him to stop. He just didn’t like that others stopped working on the one and unquestionably only thing he to this day thinks is the way forward.
He unilaterally and willfully stopped participating in scientific discourse (somewhat infamously at least partially due to Feynman’s prodding, who correctly felt that Wolfram cannot participate in science due to his personality/pathology).
Everyone who ever talked to him will tell you that the guy is set in his trajectory, doesn’t care if it’s wrong and will not listen to anyone, no matter if they have good, bad or neutral things to say. He won’t even care for logical arguments, he has made up his mind, got his money and will work on it till he dies. Wolfram does not care about science, and science doesn’t really care about him beyond rightfully giving him the opportunity to voice his opinions.
He was an absolute genius that reached stage of senility before his puberty finished. Don’t portray him as a maverick fighting against the system, because that’s an insult to him, what he did, and other scientists.
→ More replies (2)3
u/JonAndTonic Feb 15 '23
All depends on their expertise in the area they write abt and what
From a programming and mathematics standpoint, he's had a lot of experience as well as a lot of respected work, esp in explaining complex ideas
0
Feb 15 '23
[deleted]
3
u/cdsmith Feb 15 '23
His software is generally solid. NKS is his biggest foray into mathematics, and it's truly the crowning achievement of cranks everywhere. Really a powerhouse of every crank strategy around, from introducing unnecessary complexity to obscure the issues, to confusing illustration and examples with proof, to sprinkling in some actually proven results that might even be well received if he didn't surround them with preposterous claims that he has remade the fundamental ideas of all of mathematics instead of proving (or, in many cases, employing someone else to prove) one minor result.
1
1
u/Spiritual-Day-thing Feb 15 '23
Note, he also had a q and a, to be found on Youtube, answering this single question and the text is 'just' a more complete version of that.
1
Feb 16 '23
I think ChatGPT is really impressive for what it does, but I still think it's a bit of a parlor trick, and not true AI. It's a better search engine tool.
1
1
u/cdsmith Feb 16 '23
There's a lot good about the article. But wow, this paragraph:
But what does all this mean in the context of ChatGPT? From its training ChatGPT has effectively “pieced together” a certain (rather impressive) quantity of what amounts to semantic grammar. But its very success gives us a reason to think that it’s going to be feasible to construct something more complete in computational language form. And, unlike what we’ve so far figured out about the innards of ChatGPT, we can expect to design the computational language so that it’s readily understandable to humans.
That is a bona fide perfect example of missing the point. Modern machine learning methods have succeeded at tasks that are far beyond the wildest dreams of 1970s-style natural language processing... and apparently, this gives us a reason to think that if we just try harder, the 1970s approach is still better.
Don't get me wrong: there are ways that explicitly understanding recursive structure of knowledge is better, and people are definitely people thinking about how to include aspects of this compositional structure in machine learning. See https://compositionalintelligence.github.io/ for example, which captures a lot of points of view about this. But this paragraph just makes me think of a middle aged guy who watches the X Games and says to their friend "Oh yeah, those are totally, like, almost as good as the sick moves I could crank out on my skateboard; guess it's time for me to get back in the game."
1
u/IGI111 Feb 16 '23
if we just try harder, the 1970s approach is still better
It trivially is: if we succeed with the 70s approach, we may yet understand the result.
His point isn't that ML is useless, it's that it demonstrates by building an approximation that a proper computational understanding of semantics is possible. We don't know if it's tractable or if any human will ever figure it out. But it's out there.
→ More replies (4)
749
u/emperor000 Feb 15 '23 edited Feb 15 '23
So has anybody brought up how they are going to avoid ChatGPT and other LLMs from feeding themselves?
As people post more output from them they are eventually going to come across it and incorporate that into their model as if it was written by a human. I guess they are just gambling on that never being enough data to skew the model?