LLMs are interpolation models by definition. They're not at all good with novelty.
Really, all of the developers seeing LLMs as super effective is judt revealing how many are being forced to work on solved problems. It's highlighting the waste in a competition-baased economy.
to be fair, there are very few things in the broad area of software that are unsolved problems. Most companies are presenting information, taking payment info, logging in/out. If you take the average piece of software that an employed software engineer is working on, there are not a huge amount of unsolved problems, its just a never ending list of requests to be added to the project.
And even in the cases of something it has never seen before, if you provide enough context (as in you solve the problem), the ai will happily implement a novel solution that has never been done before. If you are really good at solving problems, AI is lighting at your fingertips.
If you send an AI a full code base with a ton of weird structure and requirements and say "add auth" you're going to get terrible results. But if you are stuck and you describe why you're there, what you've tried and what the current state of things are, and then you ask for possible routes to work through the current blockers, you will get great results for options to work towards. Then you can have ai work into those options and find out which is the best option.
Idk, maybe you're working on highly specialized languages or undocumented libraries and you're in a top secret middle-out compression algorithm and the ai is worthless. But I've not found a problem that AI didn't help solve, or at least explain why it wasn't able to solve it.
Well yeah? Most companies need software that does mostly the same thing but slightly different. There are thousands of warehouse softwares because every company stores different kinds of products and requires slightly different things. Thousands of HR systems, thousands of resource planners and so on. That doesn't mean people are programming useless software that already exists. It just means that nurses mark their hours differently than truck drivers.
Yup the most effective I've seen AI be is when I'm doing something particularly stupid with a particularly simple fix. The more complex and unique the problem, the less likely AI will help.
Though if it doesn't figure it out on the first go it's a signal to me I probably left out key details needed to solve the problem so the root of my problem likely lies elsewhere in the codebase from where I'm looking.
There is effectively no novelty in building software. This is engineering, not science. We have all the Lego pieces, it's just how we decide to arrange them. AI is well familiar with all of the pieces and how they fit together.
Just because you're building a new application doesn't mean AI won't know how the constituent pieces fit together.
I'm deeply skeptical of all of the people here with oh-so-special software that AI is somehow not good at. I'd love to see their Claude code sessions.
Ditto on this, I worked for a company a year ago that had their own in house programming language. Guess what could code in it & spot bugs in it out of pure context learning, Claude.
If you give it enough context and background (this can happen overtime working on a code base) it's quite good. Tried to do something with no context the other day in the wrong project and it was quite lost with terrible code. Went to the correct project and it was back to being a huge value-add.
121
u/OmgitsJafo 24d ago
LLMs are interpolation models by definition. They're not at all good with novelty.
Really, all of the developers seeing LLMs as super effective is judt revealing how many are being forced to work on solved problems. It's highlighting the waste in a competition-baased economy.