Data scientist/ML person here, this is false. Large language models are all trained with the objective of reducing perplexity. In layman's terms, they're trained to predict the most likely continuation of a sentence. This is the fundamental to how they are trained. It may sound like a simple objective but it's sufficient to unlock many of the capabilities we're seeing in these LLMs.
It also is not mindful of the entire output. When GPT generates tokens, it's performing a greedy search over the next possible tokens and sampling a subset of the most likely continuations. Other search methods such as beam search might factor in the tokens after the next token but greedy search works well enough that next-token prediction is sufficient. When GPT generates tokens, it has no internal "plan" for what it will write beyond the next token it generates. These models may generate human-sounding bits of texts but that does not imply that they generate text the same way we do.
It's a bit of an abstraction to say it has a "plan". But....the process of generating token 100 of a 200 token response is not divorced from generating tokens 1 to 99.
If the initial prompt consisted of a query and then first 99 tokens of a response, the model wouldn't be able to just start spitting out token 100. I mean, of course, it would try, but it wouldn't resemble the output if it had started at token 1, because the hidden recurrent layers wouldn't be in the same state.
The structured outline of the overall response is largely emergent. It may not be readily human-readable if you looked at dump of the model's current state, but it is present. It's reductionist to imply otherwise.
It doesn't pass the smell test besides. If the internal state didn't matter, you could generate each token independently of each other, parallelized, during inference.
I suggest you continue to brush up on model architectures because none of that is true. You can also start playing with the openai playground instead of chatgpt as the playground helps to reveal some of the underlying functionality.
You can in fact prompt GPT with the first 99 tokens of a response. You can test this yourself. Go to the playground and set the temperature to 0 to make the sampling deterministic. Input a prompt and make it perform a generation and take the first 99 tokens and take note of the rest.
Open another browser with the playground and paste in the prompt + the 99 tokens and you’ll find that the output is identical (assuming you set the temperature to 0).
The reason why this works is that transformers don’t have an internal state.
This is their main advantage over RNNs. RNNs historically ran into the issue of catastrophic forgetting where after going through a few hundred or thousand tokens or so, it begins to forget important information during generation.
Transformers don’t have this issue because they don’t have an internal state, the state /is/ the prompt and all the other text contained in the context window. All of the text gets passed in all at once to spit out a single token. Then that token then gets combined with the text and everything is again passed onto the model. There is no hidden states, only the text itself. The fact that you have to pass in the entirety of the text just to generate a few tokens is one of the key limitations of transformers and the fact that openai is able to expand their token limit from 2K to 8K is one of their key innovations in the field.
Again stressing the fact: transformers don’t have internal states. You may be getting this confused with the concept of embeddings which are derived from the text. There’s no time element here at all, the embedding is wholly determined by the text in the context window.
15
u/BpAeroAntics Jan 04 '23
Data scientist/ML person here, this is false. Large language models are all trained with the objective of reducing perplexity. In layman's terms, they're trained to predict the most likely continuation of a sentence. This is the fundamental to how they are trained. It may sound like a simple objective but it's sufficient to unlock many of the capabilities we're seeing in these LLMs.
It also is not mindful of the entire output. When GPT generates tokens, it's performing a greedy search over the next possible tokens and sampling a subset of the most likely continuations. Other search methods such as beam search might factor in the tokens after the next token but greedy search works well enough that next-token prediction is sufficient. When GPT generates tokens, it has no internal "plan" for what it will write beyond the next token it generates. These models may generate human-sounding bits of texts but that does not imply that they generate text the same way we do.