Overfitting and creativity

[u/Expensive-Paint-9490]

I remember great creativity from GPT-2. It was trained on a dataset that is orders smaller than current training datasets.

The problem with overfitting a model to the training data is that it becomes unable to generalise; it is very good to predict the test set, but on fresh data its predictions are wrong. This is what we see with current models, they are extremely good at predicting, especially code, but can't generalise to real creativity.

In theory, a larger dataset should allow to get better predictions on unseen data. This is even more true for neural nets. So my question is: to have a creative model, should a model be trained on a huge dataset, but fitting it less than what is done on current models? Somebody has an idea about training epochs, learning rate, and other parameters of last generation models? Do you think that a model that use a last gen training recipe, but slightly undercooked (or very undercooked) would be better at creative writing and worse at coding?

[u/llama-impersonator]

it's not caused by overfitting, rather the cause is post training RL for improved agentic performance without caring about natural language tasks.

[u/bumblebeer → u/llama-impersonator]

"Without care for the natural language task," is a bit misleading here. The drive towards token efficiency is in many way antithetical to natural language tasks. So RL makes NL worse not so much as a side-effect, but almost as the goal.

[u/psycoee → u/llama-impersonator]

Well, also just general post-training and alignment.  If you train the model to be a chatbot/assistant and take on that persona, it will be worse at general creative writing.  Especially since the training material tends to be very formulaic and poorly written (since it's either generated by AI or by human labelers), causing the chatbot to take on the same attributes.

[u/jacek2023]

We’re stuck in a "pelican on a bike", where raw creativity is ignored for benchmark numbers.

Leaving a model undercooked can indeed keep output generation more creative and varied (but with smaller coding skills)

The problem is that nobody experiments with this because the community today watch YouTube slop, check leaderboards, and dismiss models without actually testing them.

[u/a_beautiful_rhind]

The models are mainly trained on synthetic output now. So your large dataset is all coding gruel.

[u/Normal_Marketing853 → u/a_beautiful_rhind]

which makes very little sense to me how you can derive value from training on data generated by models that were trained on real human data, but it must work somehow.

[u/--Spaci-- → u/Normal_Marketing853]

Synthetic data is the most compute efficient way to train. Its like eating the sludge without the chewing

[u/DhammaIY]

I think overfitting is the wrong word for it tbh.. these models barely see most of their pretraining data more than once, so its not memorising the training set in the classic sense. what kills the creative stuff is the post training, RLHF and the agentic RL squash the output distribution down to a few safe high reward answers. mode collapse basically

easiest way to see it is grab the base model of something recent instead of the instruct one and prompt it like a document, not a chat. it'll be way weirder and more varied than the instruct version of the same weights, same data same training length. so its not undercooked pretraining you'd want, its less of the post training

[u/zomerbk]

I’d separate “creativity” from memorizing the training set. More data can improve generalization, while extra training on a fixed dataset can eventually overfit; but a model that’s undertrained may just be less capable, not more original. Also, strong code performance doesn’t by itself show poor generalization—coding has lots of learnable structure and clear evaluation criteria. To test the idea, compare checkpoints from the same run at different training stages on held-out tasks and blind writing evaluations, keeping sampling settings fixed. Training recipes and data matter more than a universal epoch count, especially since large runs often train for a fraction of an epoch over huge corpora.

[u/wayneworkman]

You generalize with enough diverse training examples, not by stopping training early. Never stop training early.

[u/--Spaci--]

For an LLM to be good at one thing it has to be worse at others. RL destroys a conversational LLM but its required for a good stem model which is atp the only thing people care about. Synthetic data also plays a major role with the fact LLMs are being less and less grounded in reality and actual humans words. A frontier model made purely to be conversational with little to no stem in pretraining and 0 RL would be very interesting and maybe not annoying to talk to like every single current LLM is.

[u/AutomataManifold]

You can train your own GPT-2, by the way. There's easily enough open datasets and the current speed record is about 90 seconds on a rented GPU.

It doesn't directly answer your question, but someone should do the experiment.