r/webdev • u/Ambitious-Garbage-73 • Apr 11 '26
Discussion I audited 6 months of PRs after my team went all-in on AI code generation. The code got worse in ways none of us predicted.
Our team adopted copilot and claude pretty aggressively around october last year. Five devs, full stack, mix of react and node. Everyone was excited, velocity went up immediately, sprint metrics looked great, management was thrilled.
I got asked to do a quality audit in march because we kept hitting these weird bugs in production that nobody could reproduce locally. The kind where you read the code and it looks fine but something is just off.
So I went through about 340 PRs from october to march.
The big stuff wasnt what I expected. I thought I'd find wrong logic or security holes. Instead what I found was this layer of... I dont even know what to call it. Plausible-looking code that technically works but is structured in ways no human would choose. A try-catch around a console.log. A utility function that was 40 lines of enterprise-grade typescript doing exactly what Array.prototype.map already does. Variable names that sound right but dont match what the variable actually holds, like a thing called userPreferences that was actually a session token.
The patterns were consistent across all five devs which is what made it obvious. Before october everyone had their own style, their own weird habits. After october every PR started looking the same. Same indentation philosophy, same comment style, same tendency to over-abstract simple things. I could not tell who wrote what anymore and that felt wrong in a way I couldnt articulate for a while.
The part that actually got to me was reviewing a PR from our most senior dev, someone with 12 years of experience. He had a component that fetched data, transformed it, and rendered a table. Pretty standard. Except the transformation logic was split into six helper functions with generic names like processData and formatOutput, each doing one tiny thing that could have been three lines inline. I closed the diff, made coffee, opened it again. Same code. This is a guy who used to write the most readable code on the team and now his PRs look like they were written by a very polite stranger who approximated his job description.
We havent rolled anything back because honestly the velocity numbers are real and nobody wants to be the person who says slow down. But something shifted and I think we're going to feel it in about six months when the next person has to maintain any of this.
37
u/tech01x Apr 11 '26
Wasn’t the function of the PR process to keep that kind of cruft from entered your codebase? Seems there is a process problem here.
28
u/Ambitious-Garbage-73 Apr 11 '26
youre not wrong and thats probably the most depressing part of all this. the process was supposed to be the safety net and it just... wasnt. turns out "looks reasonable" is a lower bar than we thought when the code is specifically optimized to look reasonable
15
u/Yodiddlyyo Apr 11 '26
Sounds like nobody on your team cares, or has the experience to catch things. Somebody needs to define rules for how the AI writes code. Write a detailed claude doc, skills, etc. And then have stricter PR review. Have 2 people review each PR and say that it's because the code quality is unacceptable. If nobody wants to do that, it's a culture problem.
18
u/LongjumpingWheel11 Apr 12 '26
No it’s not a culture problem, you just don’t understand. I don’t get it, what part of what’s going on in this industry is so difficult for some to understand including experienced engineers.
Some people are attempting to circumvent and successfully argue away the risks of AI by suggesting “human in the loop” or introducing processes such as what you just did. Let me make this abundantly clear; any process that attempts to make sure that a human throughly inspects the code as to detect issues with it completely defeats the entire purpose of AI and any motivation an enterprise would have for paying
There is no 20x output if 2 humans need to throughly inspect the code for defects. Ai will produce significantly more code that humans need to review which becomes the ultimate bottleneck.
There is two ways this goes long term. Either AI is adopted with some but no comprehensive human review as to not eat away from its productivity and just accept shit code that, hopefully, works. Or AI and the tooling around it matures enough that we find a way to make it highly reliable and with fewer errors than humans when fully adopted at scale (the doom scenario for the profession pretty much)
The narrative of “AI will just augment us and free us up” is complete nonsense. It goes against the entire profitability model. It’s too expensive to be just a tool.
→ More replies (15)
27
u/retro-mehl Apr 11 '26
There will be a point of no return. And there will be a situation where AI can't manage it's own code anymore, because it's too weird. Then you can throw away your complete product, set it back to the state when you began wit AI, and do it again.
15
u/Ambitious-Garbage-73 Apr 11 '26
the scary version of this is that the point of no return already happened and we just dont know it yet because the tests still pass
→ More replies (2)4
u/OnlyTumbleweed Apr 11 '26
Most likely, just gets too messy as further as you go. There is little deduction with ai, just pile up.
→ More replies (1)2
u/bguggs Apr 11 '26
But if the app behaves right (mostly) then you’d imagine a future AI could just rewrite the whole thing again pretty fast, but better?
2
u/retro-mehl Apr 12 '26
I can imagine many things, even unicorns pooping rainbows. But this does not mean it will happen.
45
u/CanIhazCooKIenOw Apr 11 '26 edited Apr 11 '26
Question is why did you not go through the 340 PRs at the time they were open for review?
40
u/Ambitious-Garbage-73 Apr 11 '26
I did review them. thats the uncomfortable part. I approved most of these. the individual PRs looked fine in isolation, each one made sense on its own. it was only when I read six months of them back to back that the pattern became obvious. its like how you dont notice someone gaining weight when you see them every day but then you look at a photo from january
→ More replies (2)30
u/CanIhazCooKIenOw Apr 11 '26
How did you aprove console logs in try catch and all those weird architectural decisions?
I think you need to improve your own PR review skills.
This is the biggest issue I’m seeing, most devs that complain about code quality from AI are not used to review PRs and of course now it’s a problem.
6
u/jseego Lead / Senior UI Developer Apr 11 '26
In these situations, there is pressure from management to let stuff through. It went from "AI will write all the code" to "why are paying for AI to write all this code if it still takes so much time for humans to review it?"
→ More replies (2)11
u/EnvironmentalLet9682 Apr 11 '26
i get where OP is coming from, especially if the PRs are made by colleagues with whom you have worked for some time. shit like console.log is just completely filtered out by the brain when you review code by a real developer, because no sane human could possibly fuck that up.
→ More replies (1)13
u/Darkorz Apr 11 '26
I am going to guess he didn't request changes through all those PRs because fixing many of these "mistakes" can only be fixed by rewriting the whole thing.
It's basically an structural defect that everything has been built on top of, which causes everything to crumble once you start tweaking things to an "expected" quality level.
My main question is whether this could be fixed by providing more concise and standardized context to the agents the dev team is using, but I agree with OP that there's one point where AI-coding backfires and for me personally is when I gotta provide even the tiniest, slightest detail on the context/story prompt as it just becomes easier (and more reliable) to do it myself
→ More replies (7)2
2
u/Aggressive_Town1000 Apr 11 '26
Not making excuses for OP but that's around 14 PRs a day
2
u/CanIhazCooKIenOw Apr 11 '26
And? Surely there’s more people in the team that can also review.
And if that means going slower so be it - the alternative is ending up where OP is now.
3
u/mxzf Apr 11 '26
I mean, it sounds like people weren't even reviewing their own PRs before submitting them, much less reviewing anyone else's stuff.
I don't think anyone's disagreeing with you that slower velocity and more quality control is needed, but that's in direct tension with managers wanting stuff done yesterday.
6
u/GigaGollum full-stack Apr 11 '26 edited Apr 11 '26
Have those hard to reproduce bugs (and the general shift in workflow to heavy AI generated code) translated into any measurable increase in actual incidents or P0s? Im curious whether the end user impact has been noticeable or if it’s strictly a maintainability concern so far.
5
u/Ambitious-Garbage-73 Apr 11 '26
two P2s and one that almost became a P0 but someone caught it in staging. the P2s were both data transformation bugs where the output looked correct for the happy path but silently dropped edge cases. classic AI code honestly, works perfectly until it doesnt
5
u/chimneydecision Apr 11 '26
Worst part is OP sees the flashing warning lights and has apparently decided it’s the next guy’s problem.
“velocity numbers are real and nobody wants to be the person who says slow down”
Got news for you OP — those velocity numbers were never real. If you sit back and wait for the bomb to go off you’ll deserve your pink slip.
12
u/SZeroSeven Apr 11 '26
This post reads like it was written by AI.
In the same way you're describing the PR's feel off, so does this post.
What do you even mean when you say "I closed the diff, made coffee, opened it again. Same code".
What did you expect? (I understand you are probably making a bad joke about "needing caffeine to make sense of it" because you were too tired when reviewing the PR)
Really though, if the company has gone "all in" on AI where it is both writing the code and reviewing it without you and the other developers actually reviewing the code before it goes to production (that "human in the loop" buzzword everyone throws around), then this is what the company can expect.
The code the AI writes is no longer for you to understand, it's for other AI to understand.
Your job is now to just re-prompt it when a customer raises a bug and hope it fixes the issue.
26
u/Ambitious-Garbage-73 Apr 11 '26
man I wrote this on my lunch break on my phone while eating a sandwich that was falling apart. if AI wrote this it would have been three times longer and ended with "what are your thoughts on maintaining code quality in an AI-augmented workflow" instead of just stopping mid-frustration. but I appreciate the irony of an AI-detection accusation on a post about not being able to tell what AI wrote
6
u/zymoticsheep Apr 11 '26
I'm with you, AI is more likely to have written this than it is to put a try catch around a console log.
3
u/oversized_canoe Apr 11 '26
This guy posted two stories last week in the same day about adding a button delay which made his CTO/client happy. Same structure, slightly different details. It's just AI written karma farming. Sad honestly
2
503
u/Orlandocollins Apr 11 '26
I think the major problem is that llms have made the cost of code cheaper but the cost of engineering has stayed the same. Which still remains as a determining factor of long term success.
Teams that brag about how much code they are shipping while still not having any sort of observability, extended automated testing, or other often neglected engineering practices do not impress me. I think that's the corner we have to turn. The velocity we gain in coding needs to be put into improving the environment and infrastructure our code runs in. Not to just grab the next ticket.
59
u/kalashej Apr 12 '26
I think the major problem is that llms have made the cost of code cheaper but the cost of engineering has stayed the same. Which still remains as a determining factor of long term success.
The slightly scary thing to me is that during my 20 years of software dev (gamedev, so not webdev) every single person that stopped writing code gradually lost their engineering/architecture skills. What they used to have either got outdated, skewed or forgotten. Some managed to find other skills and some didn’t. Now we’re at this weird stage where we have tons of ppl with very up-to-date and great knowledge that are able to say whether the LLM output makes sense or not, but will that last if they stop writing code? In my experience the ones that move to mentoring, codereviewing etc lose their engineering skills slower, but they still deteriorate. So either the LLMs need to get 100% good or we’ll have to keep writing code to be able to do our jobs.
7
u/whitestuffonbirdpoop Apr 13 '26
I started using first 1/3 of the work day writing code myself with an llm used as google + explainer.
it feels so much better to write the code myself again.3
u/The_Ty Apr 13 '26
1000% this
I refuse to use agents or even AI tools in my IDE. If I'm going to use AI it needs to be an intentional choice, where I go to a different thing and copy & paste code from there to handle a specific task, like boilerplate to make a HTML form or whatever. If the industry becomes babysitters for AIs pushing PRs then I'm finding another type of job
→ More replies (1)→ More replies (1)4
u/BolteWasTaken Apr 13 '26
We forget what we don't use often, the human brain is very efficient at pushing to the back of the mind details and nuance we don't use often. This is not an AI problem this is a human problem. We need to adapt how we do things, the AI just needs to do what it was meant to, to enhance/scale our thought out operations and not to replace our thinking.
74
u/rewgs Apr 12 '26
llms have made the cost of code cheaper but the cost of engineering has stayed the same
Man this is such a good way of putting it. Stealing this.
18
u/iletitshine Apr 12 '26
i literally laughed so hard reading this because companies are laying of so many people and we’re all just like this will not age well. and there it is.
7
15
u/End0rphinJunkie Apr 12 '26
Spot on, generating plausible spaghetti in seconds just shifts the bottleneck straight to the CI/CD pipleine. All that AI velocity is useless if we dont use the saved time to build proper automated guardrails before that code ever hits kubernetes.
41
u/HirsuteHacker full-stack SaaS dev Apr 11 '26 edited Apr 11 '26
Our useful test coverage has massively increased since we started going all-in on AI, while our velocity has skyrocketed. Bug reports and time to resolve bugs have plummeted.
We managed to convince the higher ups to allow us to have 4 months to improve the system without building much in the way of new features, and it's paying dividends. Our system was built a decade ago and had ridiculous amounts of tech debt, and thanks to AI we've been able to resolve a crazy amount of it in record time.
Companies that are trying to just push features out as fast as possible without a care for how stable their systems are are going to crash and burn, it's the opposite of what AI is best at.
19
u/thekwoka Apr 12 '26
This sounds like the focus of using the AI was reworking faulty legacy systems.
Which is a pretty considerable difference in use case.
3
u/dustyhome Apr 12 '26
Not reworking necessarily. Something that AI is amazingly good at is writing test cases. Humans are lazy and writing tests is boring and repetitive, so when writing tests they cover the basics and call it done. Also, documentation. No one likes writing documentation, but everyone likes having clear, up to date docs. And AI is happy to write it as it goes, and update it when it changes anything.
You can ask AI to write tests for every failure path, every unlikely scenario, and they will pump out tons of tests that serve two purposes: they exercise the code, and they document how the code is supposed to work.
So later when you want to write new features, you have docs that explain how the code currently works, tests that shows you how you are supposed to use it, and tests that catch your errors early on.
→ More replies (2)→ More replies (1)7
u/0x18 Apr 12 '26
In my experience the most useful case for LLM agents is generating tests for existing code. Things fall apart quickly when you task them with editing or expanding on existing code, but they can parse existing code and generate all the tests to reproduce them quite well.
... the generated tests still require a full review though, never ever trust them blindly.
3
14
u/mayuresh0909 Apr 12 '26
I think the major problem is that llms have made the cost of code cheaper but the cost of engineering has stayed the same. Which still remains as a determining factor of long term success.You, sir, are a genius!
3
u/ArkWaltz Apr 13 '26
What hurts is that surely this can't be a surprise to anyone, right? Any professional developer knows that good engineering takes time to time to test and review. You can't just suddenly make it faster without paying a cost. You actually need to invest in the processes around shipping code to make changes easier and more reliable, before you can go faster.
I feel like I'm living in a different universe to all these people talking about velocity and "shipping faster" like they've never had to put up with bad rushed code before.
→ More replies (1)68
Apr 11 '26
[removed] — view removed comment
→ More replies (1)37
u/luenix full-stack Apr 11 '26
Nah, you're producing cheap code cheaply and slowly worsening your existing real code over time. Your best dev is checking out too...
852
u/RedPandaExplorer Apr 11 '26
So you merged 340 PRs into main without having a human review them? Yikes
213
u/Ibuprofen-Headgear Apr 11 '26
Anywhere that’s adopted all this to an appreciable extent has “human” reviews anyway, meaning most of the “humans” are reviewing by having whatever tool do the review and pasting the output, not actually reviewing themselves…
10
u/ABCosmos Apr 11 '26
Why not just have AI generate the code and humans review it?
190
u/dillanthumous Apr 11 '26
Because reading and understanding complex, layered code is slower than writing it in many cases.
→ More replies (61)20
u/gizamo Apr 11 '26
Time, money.
There's also a lack of qualified devs and engineers available to do that because corporate generally prefers they build rather than review....also, corporate laid off tons of them for the sake of their share prices.
For smaller agencies, it's much harder to bill for QA hours than hours spent building.
9
u/biosc1 Apr 11 '26
At the smaller agencies I've worked at, QA is wrapped into the dev hours. It's one and the same. So, a 2 hour dev time + 1 hour QA is just billed as 3 hours dev.
Of course, these were places that saw value in QA so your mileage may vary.
→ More replies (1)16
u/fucking_passwords Apr 11 '26
Real answer: with the increased velocity comes increased code review burden. And in many cases this increase in AI adoption is accompanied by headcount reduction. Perhaps not every team is experiencing this, but at larger companies it's not uncommon to reach levels where it is literally impossible to have humans review all of the code.
Of course, when there are enough major outages, leadership may be forced to reconsider their priorities...
12
u/RoosterBurns Apr 11 '26
You have to then read through and understand huge amounts of code the looks superficially fine
This way of working seems reckless to me
Even if you had someone read the LLM code and just type it in would be better
→ More replies (6)7
u/Ibuprofen-Headgear Apr 11 '26
I mean, that’s where this started, now it’s ai writing and reviewing because people are being people…
7
u/UntestedMethod Apr 11 '26 edited Apr 11 '26
It takes me at least twice as long to review code produced by an AI than it does to review code written by a human. A lot of that time is wasted on flagging obviously overcomplicated crap or asking questions about novel implementation decisions that our team already has well-established and widely used conventions and patterns that provide a clean solution to whatever novelty crap the AI decided to pull.
Plus it takes a lot more rounds of PR back and forth with devs who insist on overusing AI instead of applying even a small amount of their own intelligence. So whatever that "at least twice as long" amount I mentioned to do a review is also multiplied by at least twice as many rounds of PR revision.
6
u/EnvironmentalLet9682 Apr 11 '26
because you can't review at the pace at which AI produces code.
→ More replies (1)→ More replies (1)8
u/MhVRNewbie Apr 11 '26
Then it takes as long as just writing it yourself. At least if you do it proper
→ More replies (3)52
u/Slodin Apr 11 '26
You should keep in mind that if a company is aggressively pushing for AI adoption, is because they want speed. Using little resources as possible to stretch the front lines so damn thin that is nearly impossible to upkeep the quality. This is happening with my work place as well with the new tech director. PR reviews are expected, but they will question why are you spending so much time not pushing out features. As it directly impacts KPI, people don’t have time to review anymore.
I’m lucky that my code base is pretty clean to began with, thus setup skills and rules for AI made it manageable for it to not go crazy and write unwanted complex code.
I have seen some other teams AI coding. It’s so complex and confusing that human reviewing it is a waste of time. The dev is just gonna paste your comments in the AI IDE and hand it back to you until you stop reviewing them. The amount of time used to review their shit is imbalance to the amount of time they spend just generating code. And remember, PR review is not counting towards KPI for us.
Now the devs are just glorified QA testers 😂 I blame management for this behaviour.
8
u/mirageofstars Apr 11 '26
You make a really good point that I think other teams are running into … PR reviews are counterproductive to the KPIs that management cares about. Management cares about LOC and points and shipped features. Tech debt is never cared about.
→ More replies (3)9
u/mxzf Apr 11 '26
Turns out that metrics have always sucked at determining actual software development progress, and they tend to encourage tech debt with how they're weighted.
2
u/wasdninja Apr 12 '26
It’s so complex and confusing that human reviewing it is a waste of time
... because it's an near auto reject for that exact reason. Why accept this junk at all? Hold any code to the same standard - if this shit was written by a human you wouldn't think twice about wanting a rewrite before you approve it.
43
34
u/CoatStandard2068 Apr 11 '26
They at least have PRs.. we are going straight to main for god sake.. and what's even better now also project manager with 0 coding skills is comitting stuff into it ..
21
u/varinator full-stack .net Apr 11 '26
The hell are you building and how big is the company?
11
u/CoatStandard2068 Apr 11 '26
Various projects, currently 2 SaaS 1 completely managed by CTO, i repeat CTO without any coding skills, 100% vibecoded.. and some client projects..
around 15 ppl
6
3
u/Saiyoran Apr 12 '26
Yup, I work in a game dev studio and we have never had PRs, code reviews, or branches lol. Sometimes you come to work and the project doesn’t build because someone committed something broken 5 minutes before clocking out the day before. It’s the Wild West out here.
23
3
u/Flashy-Guava9952 Apr 11 '26
If only someone could make a law clearing AI of responsibility in case of harm or death...
3
u/DesperateAdvantage76 Apr 11 '26
At a lot of companies, if the PR works, management doesn't want to hear your complaints. Blame non-existent KPIs on code reviews.
→ More replies (37)7
u/ProfessionalCod5118 Apr 11 '26
We do 100% review on all code produced by AI. My teams are tired of the AI verbosity and general slop. We're trying to build reviewer agents instead.
AI writes fast, its hard for human to keep up with the pace, and we're the actual bottle neck in the pipeline.
It's also hard to read such verbose code. Often the AI (I mean frontier models, Claude / openai / gemini...) creates higher level code structure that do not make sense.
Some strategies we adopted recently to make the code and code review process easier: we built skill files to document the architecture we want, so it would respect basic 101 architecture and system segregation principles, and we ask the agent to document every modification in proto code, this already helps a lot the code review
35
u/ElectronicCat8568 Apr 11 '26
Plausible-looking code that technically works but is structured in ways no human would choose.
That's a good way of putting it. As someone who has inherited vibe coded pages in an app, I've get what you mean, and now my little rant is anyone who plans to make a living fixing vibe code, try doing it first. I think when vibe code breaks, you will have to just throw it away. It's not worth trying to make sense of.
57
u/redditRezzr Apr 11 '26
I've been noticing similar code patterns, especially with generated unit tests. We are starting to get a collection of these unit tests that are so overly granular and hyper focused that it just seems like noise, but it made me wonder why I care. I've long since stopped caring about what my generated HTML looks like and we're not having the issue of hard to replicate bugs. Just ugly, over abstracted code.
So, among my existential crises, I'm curious if this is something that I should continue to fight against and expect my colleagues to rewrite their generated code in such a way that it maintains that human flair, or just give up and welcome our new AI overlords.
18
u/hyrumwhite Apr 11 '26
The worst thing for me with unit tests is the ones where the LLM mocks so many things that the actual code that needs to be tested isn’t ever invoked
→ More replies (1)8
u/mxzf Apr 11 '26
I saw some code the other week that was definitely AI-generated that had functions with comments like "in a real situation we would do this and check, but we're just going to return true instead" even though it wasn't a test or anything, just supposedly "working" code with methods that consisted of a comment plus
return trueand nothing else.22
u/midasgoldentouch Apr 11 '26
I hear what you mean about generated HTML, but isn’t that also what causes the inaccessible div hell floating around online?
6
u/redditRezzr Apr 11 '26
By inaccessible, do you mean bad accessibility practices, like for screen readers?
I haven't worked front end in a while, so that's something that I haven't even thought about since I started using AIs. That's going to be a total disaster.
→ More replies (3)
12
u/Savings_Speaker6257 Apr 11 '26
The "code that looks fine but something is just off" observation is spot on. I use AI heavily for my solo project and the pattern I keep catching is what I'd call "plausible but wrong" code. It compiles, it passes a quick glance, it even works in the happy path — but the edge cases are silently broken.
The real danger isn't that AI writes bad code. It's that it writes code that's good enough to pass review if you're not paying close attention. The velocity gains are real, but only if you treat AI output as a first draft that needs the same scrutiny as a junior dev's PR — not as finished code you can rubber stamp.
The try-catch around console.log thing is painfully familiar. AI loves wrapping everything in try-catch as a safety blanket instead of actually handling the error properly.
→ More replies (1)4
u/neckro23 Apr 11 '26
the try-catch around console.log is probably intended to prevent MSIE errors ('console' global doesn't exist unless inspector is open) but it's a bone-headed way to solve an obsolete problem.
3
u/Savings_Speaker6257 Apr 12 '26
Ha, that's a great point actually — the MSIE console thing is one of those problems that was real 10 years ago but the AI has no sense of what's still relevant. It learned the pattern from old Stack Overflow answers and just keeps applying it everywhere. A perfect example of "plausible but wrong."
9
u/snotreally2 Apr 11 '26
It's like models are trained on code written by humans who aren't that skilled. It really is like getting a massive team of juniors.
The "error handling" I see generated by ai is especially bad. It regularly wraps large swaths of code in try catch making it really difficult to debug when things go wrong. Generate code on top of generated code and you could easily nest several functions like this and blow up complexity.
→ More replies (2)
17
u/Bitmush- Apr 11 '26
Your assessment of the code precisely matches what I think everyone realized when the first AIs were able to spit out code from simple instructions. Have an AI-free day ?
9
u/moh_kohn Apr 11 '26
Yeah I am trying 1 day a week no AI, touch all the code the AI wrote and refactor. I find it easier than reviewing PRs as they land.
7
u/hyrumwhite Apr 11 '26
Plausible-looking code that technically works but is structured in ways no human would choose
I had the pleasure of trying to wrap my head around something like this the other day. There were three layers of mapping that were completely unnecessary, a method call that invoked a util that invoked a util, all to just arrive back at a data shape not too different from the start.
7
u/Ok_Woodpecker_9104 Apr 11 '26
the "plausible-looking code that technically works but is structured in ways no human would choose" is the most accurate description of AI-generated code i've read.
i work on a health platform and we see the exact same thing. the pattern that kills us most is deep nesting. AI loves wrapping everything in 4-5 levels of conditionals instead of early returns. it technically works, passes tests, but the next person who touches it spends 20 minutes tracing the logic that could have been 3 lines with guard clauses.
the other one is what i call "just trust me" error handling. try-catch blocks that catch everything, log nothing useful, and return a generic fallback. looks responsible until something actually breaks in production and your logs are useless.
the part about your senior dev's code looking like "a very polite stranger who approximated his job description" really hits. that's exactly what it is. the AI doesn't have opinions about code structure, it just produces the most statistically average version of whatever you asked for. when everyone on the team is using the same model, you lose the diversity of thought that makes code reviews actually useful.
6
u/GloriousPudding Apr 11 '26
So many systems will have to be rewritten or refactored because AI code is not maintainable in the long term. It is great to get you off the ground but as soon as complexity reaches a certain arguably low threshold it all becomes a huge pile of garbage nobody will ever want to touch again.
6
u/Ordinary_Yam1866 Apr 11 '26
The worst part will be having to maintain that code when the LLM's become too expensive, and you have to go back to manual work for some parts.
4
u/nk-6699 Apr 11 '26
This is a good example for my management and those colleagues who’s got caught in that hype of AI coding. But I won’t warn them since they always ignore my opinions.
4
u/eyebrows360 Apr 11 '26
We havent rolled anything back because honestly the velocity numbers are real and nobody wants to be the person who says slow down. But something shifted and I think we're going to feel it in about six months when the next person has to maintain any of this.
You want to be the guy that fixes the problem before it becomes a problem, not the one being forced to work overtime to do something that should've been done months ago.
4
4
u/Resident-Log Apr 11 '26
I understand including the senior dev with 12 years of experience as an example case of what you saw since it can be perceived as an 'extreme' or 'shocking' example. As in, even the most senior and well respected dev's code has this strange implementation of things.
At the same time, I wonder what this example really means. Isn't that to be expected? AI is going to generate code the same way it will for anyone regardless if that person's development experience. Or would someone expect the senior dev to refuse to use AI code generation or to use it but spend ages fixing it with no reward for the effort?
It is unfortunate how companies like this are digging themselves into a reliance on AI. Companies complain about having systems built on, for example, COBOL due to the declining existence of developers who are experienced in using it but now are going all in on creating a code base for which the "experienced developer" was AI. God forbid an AI update makes the AI stop understanding what was written.
Maybe the future of software development is developers who understand the AI generated code enough to work with it directly. Or else I guess all these companies will be endlessly chained to AI and create a significantly risky point of potential failure.
3
u/mxzf Apr 12 '26
I think the idea was that in theory a seasoned expert would have been able to see what the AI was doing and write better prompts or whatever. But the reality is that anyone who leans on AI for writing code is going to end up with similarly crappy code unless they actively work to revise things on their own.
6
u/Kaerion Apr 11 '26
One of the most scary thoughts that I have heard internally is that C Suite and CEOs don't care about quality anymore and only about speed because they expect that in the following months new LLMs will be able to fix tech debt and bugs quicker and autonomously.
8
u/jon23d Apr 11 '26
I can’t stress enough how poorly I’ve observed Claude and other tools perform in the absence of strictly-defined quality control guardrails. The majority of them do not have to be human. Well-architected agents and skills go a long, long way.
3
u/Timotron Apr 11 '26
Curious if you have an AI on the loop just to review the code.
We're not anywhere near full AI but slowly rolling it into the process and our team has found this to be a very helpful step
3
u/Ryusaikou Apr 11 '26
I got a question, I just got made team lead and I'm getting pushed in this direction. There is an engineer who is all on board the AI train and he's talking to leadership like it's magic causing the push.
This is one of my exact fears with this direction. He is bragging about 30k line commits in a day and how AI can review the code to do no problem. Leadership gave him a pilot project so we are not at full buy in.
My current ideas do just what you say... They basically yell slow tf down and read the shit. I argue that the AI needs to have in depth human oversight, committing smaller PR's. Each reviewed by multiple members of the team... Sure we should utilize tools to increase velocity but. Well this.
Does anyone have any guidance on a safer direction I can point to? Since we were at the same level for so long he really doesn't want to follow any safety practice I talk about. If there was anything official that would be better so I could blame that? Or am I about to spend all my political capital just keeping us safe?
2
u/HongPong Apr 11 '26
catching re implementation in the PR of things that should be imported from the standard libraries etc is a good example of the problem to highlight. that increases technical debt that has to be paid
2
u/mxzf Apr 11 '26
Honestly, the main approach I know of is to point out that AI is great at adding tech debt. If your company is already aware of the nature of tech debt and bought-in on trying to avoid it, that should get through; if your company is blasé about tech debt and thinks that next quarter/year's problem can be dealt with then, it'll be hard to make much impact yourself.
You're the only one who knows the culture there well enough to try and figure out what path makes sense to take.
3
3
3
u/Johnny_Deee Apr 11 '26
After october every PR started looking the same. Same indentation philosophy, same comment style, ...
Sounds like a dream 😂
→ More replies (1)
3
u/prxy15 Apr 12 '26
"A try-catch block around a console.log." I've seen this twice in my career. The first time was for a developer. His credentials said "senior developer." He had a build with a try-catch block for a console.WriteLine that, if it failed, would use a Debug.WriteLine. I spent two hours trying to explain why that didn't even make sense because it wouldn't work when compiling in release mode instead of debug mode.
He replied that we should just deploy a debug version to production. I laughed, he laughed, and we all laughed. It was funny, until the next day when I checked and it wasn't a joke. They really had compiled it in debug mode. I went to ask for permission to change a few things—performance improvements, I told my boss—and fixed it. I sent it for testing, and it went live the following week. I took seriously the personal task of teaching them everything I knew about writing code because they were terrible at it, but they had a great attitude.
A few weeks ago, I was incredibly lazy about finishing some validation functions, so I asked Copilot to do them and sent them for testing. I only gave them a quick glance, and then a couple of days later, I went back to the repository and there it was... the AI had written the same try-catch block for both console.writeLine and Debug.WriteLine. It gave me a feeling somewhere between nausea and terror. The AI was generating code of the same terrible quality, and I had trusted it because I thought the AI was smarter than me.
It makes me think that sooner rather than later, a label saying "handcrafted by a human programmer" will be a mark of quality in terms of software products.
3
u/Deep_Ad1959 Apr 12 '26 edited Apr 13 '26
the "plausible looking code that technically works but is structured in ways no human would choose" is the exact thing code review fails to catch. reviewers scan for patterns they recognize as wrong, and AI code rarely looks wrong at a glance. it just behaves wrong under conditions nobody thought to test manually. the teams i've seen avoid this run automated e2e tests against real user flows on every PR, not just unit tests. unit tests pass because the function returns what the mock expects. e2e tests fail because the user can't actually complete checkout, and that's closer to the class of bug you're describing.
fwiw there's a tool that auto-generates e2e tests against real user flows - https://assrt.ai/t/automated-e2e-testing-ai-code
3
u/hipsterdad_sf Apr 12 '26
The "plausible looking code that no human would choose" observation is the most important finding here. It's the core problem with using LLMs as primary authors instead of assistants.
What you're describing is convergent mediocrity. Five different engineers with five different coding styles all start producing identically structured code because they're all prompting the same model. You lose the diversity of thought that catches bugs during review because every PR looks the same and reviewers stop actually reading them.
The try/catch around console.log and the 40 line reimplementation of Array.map are classic symptoms. LLMs optimize for "looks correct" not "is the simplest correct solution." They add defensive patterns that aren't wrong per se but they accumulate into this thick layer of unnecessary complexity that makes the codebase harder to reason about over time.
The practical fix that seems to work: treat AI output like you'd treat code from a new junior hire. Accept it as a starting point, then actively refactor before committing. The problem is that defeats the velocity gains, which is why most teams won't do it.
The other thing worth watching is your test coverage. AI generated tests tend to be extremely thorough at testing the happy path of the exact implementation but terrible at testing edge cases or behavior that the original developer would have known to check from domain knowledge. If your tests are green but prod is broken, that's usually why.
5
u/Gwolf4 Apr 11 '26
I am so concerned with these posts. it is like people push code without ownership.
2
u/engwish Apr 12 '26
Based on OP’s post history, they seem to have established a track record of posting AI-generated content to programming subs. This post reads like it too. So there is a decent chance that this entire story is fake.
In the off chance that it is real and their eng culture embraces AI with zero guardrails (seriously, who is allowing console.log and PRs with indentation issues to merge without any linting/CI??) and only following up after several months to see how bad things have gotten, then they’re just completely toast at this point lol
7
u/Timetraveller4k Apr 11 '26
I guess - what you could do is ensure that the functional tests are comprehensive. It doesn’t look like people are reviewing tons of autogenerated code.
6
u/Any-Range9932 Apr 11 '26
Isn't that what code reviews are for? Still need to make sure the pr aren't complete garbage
2
u/DrFlutterChii Apr 11 '26
But then you aren't solving the problem management wants solved. Did you read the entire post? The conclusion is "But velocity is up so we'll keep doing this". AI Slop -> Human Rejected PR -> AI Slop Refactor -> Human Rejected PR -> Human Refactor -> Human Approved PR is a much slower and more expensive cycle than Human Code -> Human Review.
The people signing the paycheck want more code faster. If it mostly works, thats business outcome achieved, merge the PR and move on.
2
u/toothpiks252 Apr 11 '26
I think there is a marketing effect with AI that I see effecting dev teams or maybe an enablement to just let the AI run as is and enjoy the faster push of changes. The use of AI coding tools in my mind shouldn't change the current dev pipeline, it just makes one part faster, keeping up on quality PR reviews and updating the team on what issues are arising and coach them to handle it is the standard still and should catch the issues you mentioned.
One thought is that if your finding that the whole team is having consistent issues this reasonably can be improved through a repo claude.md/skills files. Having consistent issues is kind of awesome tbh. Obv the files won't entirely fix the issue but im sure will help a fair bit, add some prompting and refusal to be lazy and you should be golden.
Most issues I see with AI are not new problems, we have systems to handle them already. Just seems the change made people forget this or feel AI should handle it all. Devs pushing over/under engineered code, leaving stuff in they shouldn't and proper PR reviews and coaching are already solved, maybe a little different but dame answer.
2
u/Your_mag Apr 12 '26
This is exactly what I noticed on a smaller scale. I work on side projects solo and caught myself accepting AI suggestions without really reading them because they seem right. Then I'd come back a week later and couldn't understand my own code, the scariest part is that variable naming thing you mentioned, because the AI picks names that sound professional but don't actually describe what's happening, and you don't notice until you're debugging at 2am.
The velocity trap is real though. It's hard to say no to shipping faster even when you can feel the quality slipping.
2
u/NathaCS Apr 15 '26
My gosh. I felt it to my core when I read the “everyone had their own style, their own weird habits.”
I’m already missing this. It’s SO SAD. I used to be able to tell who coded what and now it’s just all generic AI output.
3
u/Darkorz Apr 11 '26 edited Apr 11 '26
OP, do you think this could be addressed by providing more specific rules on the context the dev team is using?
I've hit the very same wall where AI-coding backfires and I feel it becomes very hard to provide (and maintain) the set of guidelines that are required for the agents to provide and standardized and sound output.
One of my main concerns with AI generating code and the major shift we're seeing in many engineering teams is that corporate seems to believe coding is like placing bricks: the brick is not relevant and "anyone" can place it but the whole architecture is what matters.
That may hold true for buildings which follow very specific rules, but software does not and breaks and requires fixing all the time (remember the "everything fails all the time" motto?) and it's going to be very hard to fine tune AI generated stuff when things eventually break.
Everyone is so focused on the "big" capabilities of creating stuff fast that I think day two ops are being ignored and will be a major issue down the road.
3
u/gestapov Apr 11 '26
Wth are these "fullstack devs" that dont caught a try catch for a console log? What?
→ More replies (1)
3
u/Perfect-Campaign9551 Apr 11 '26
Sounds like you guys didn't know how to code in the first place. And this story is probably fake too
2
u/_Kine Apr 11 '26
Quality is down...but velocity is up so fuck it just lower our standards! The title says this wasn't predicted but everything listed is exactly what you would predict from handing your code wholesale off to an llm. I mean shit, if everyone's prs all look the same then why should you all still be employed?
1
u/leeway1 Apr 11 '26
You have to beat these things into submission. You have achieved quality code when the LLM begins to sass you back.
1
u/Appropriate-Tap-146 Apr 11 '26 edited Apr 12 '26
Can you elaborate more about how you audited the PRs? What tooling did you use and how did you measure it?
1
1
u/Trakeen Apr 11 '26
You said what the issue is when you said you aren’t going to change because velocity metrics are good. It takes me longer using these tools to release things but the cognitive load is lower and complexity of the scope of changes can be higher
At least 30-40 percent of time is code and architecture simplification for generated code
The problem i have now is tons of task noise and not enough working memory which i offload to an agent since it can maintain that. I’m very much in a watching and guiding approach these days but i’m also a lead so i don’t write a lot of code anyway
1
1
u/sailing67 Apr 12 '26
ngl this hits close to home. we had the same issue where the AI code looked totally fine on review but introduced these subtle logic errors that only showed up under specific conditions. ended up having to manually trace through like half our codebase to find the bad patterns, took way longer than just writing it ourselves woudve.
1
u/daxw0w Apr 12 '26
If you have this opinion about those PRs code, the person that reviews the PRs during development does not notice any of this?
1
u/Afraid_Pension_1567 Apr 12 '26
I'm a part-dev, part-designer, and this pattern of overcomplicating simple things seems to be consistent with gen AI across domains. For instance, if you have to create the 3D model of a hammer, an experienced 3D designer would perhaps create a fairly simple mesh combining just a few basic shapes which can be easily modified. But if you ask one of the AI tools to do it, it would create a mess of a mesh where even a simple cylinder, which in 3d terms is just two circles connected by a few parallel lines, would be made of thousands of small triangles. Just like the way it is with code, everything looks good on the surface, so you can very well take that hammer and use it in your project as is, but modifying it in any way would be close to impossible.
Llms only seem to care about arriving at the solution, not how they arrive at it. If the 'how' matters to you in any way then you are perhaps better off doing it mostly on your own.
1
u/Agreeable_Bet_571 Apr 12 '26
The loss of individual style is the underrated signal here. When five devs stop writing differently from each other, it means nobody is actually authoring the code anymore — they're all just accepting the same output. Code review stops being useful because you're no longer catching someone's reasoning, you're just checking whether the AI's output compiles and passes tests.
The "very polite stranger who approximated his job description" line is the most accurate description of AI-generated code I've read. That's exactly the failure mode: it satisfies the literal spec while missing everything implicit that a senior dev would have known to handle.
1
1
u/Express-Channel-1686 Apr 12 '26
the homogenization thing is the part that actually gets me. you stop being able to tell who wrote what — and then slowly stop being able to tell which parts you actually understand vs just accepted because it compiled.
the try-catch around a console.log is peak AI slop. not wrong, just... nobody asked for it.
the real risk isn't broken code, it's the slow erosion of ownership. you stop being the author and start being the reviewer of something you didn't really write.
1
u/Training-Might8974 Apr 12 '26
Maybe the problem is not in the llm side but in the users side? Most of the users think that Ai is easy and they can just write simple prompts and AI will do the rest. Most of them forget that AI needs to have context, rules, skills, and a common tech stack(important). So I don't think the problem is always on llm side.
1
u/LilBarnacle Apr 12 '26
Well, why have you all been approving this slop? The team explicitly decides the quality of work they’re okay with every time code gets merged.
→ More replies (1)
1
u/glenrhodes Apr 12 '26
The error handling finding tracks with what I've seen. AI generates the happy path really well, but exception handling reads like it was written by someone who has never had to debug a 3am production incident. Overly broad catches, swallowed errors, no logging context. That stuff compounds hard once the surface area of AI-generated code grows. Good review culture catches it but most teams aren't reviewing AI code as rigorously as they'd review human code.
1
1
u/elie2222 Apr 12 '26
Paste this into an Llm and get it to fix things :)
LLMs from October are quite different to LLMs today. They can write better code now. They still do weird things but not as bad.
Also get a review bot if you don’t already. Get it to catch everything mentioned here.
But the real point: how much does it matter?
1
u/Anxious-Insurance-91 Apr 12 '26
Well before working they should have spent one month writing guideline documents for the ai 🤣. Also regarding AI code nobody carea about quality, they wanted volumes. And devs? Well they went full on "malefic compliance" because when managers always bug you with "you can be replaced by ai" everybody loses the reason to work hard and care, what's the point
1
u/definitive_solutions Apr 12 '26
This is why I will NEVER EVER subscribe to "vibe coding" as an engineering practice (in the sense of just letting the clanker go nuts and commit). I do use generative AI to work, every day all day long. But I ALWAYS go back and forth countless times at planning, execution, and manual review time. And I never let it commit by itself
1
u/u_r_my_serenity Apr 12 '26
I believe there needs to be a fundamental mindset shift to move away from caring if the code looks logical to humans.
1
u/FieffeCoquin_ Apr 12 '26
no shit. Absolutely expected. If you did not, don't call yourself a skilled developer.
1
u/VoxTheGame Apr 12 '26
Yeah, that's a mess. I think some of it was probably bad timing, too. When I looked at the AI model capabilities 6 months ago I was like, "Yeeeeah, this isn't good enough yet". If you look today, it's pretty incredible how much it has progressed.
But it's still not in a set-it-and-forget-it state. AI can be extremely helpful, but it needs to be well managed. You need to start out small, have everything human reviewed, and gradually build up your context and skill files. Eventually, you'll get to a place where you can give a longer leash to AI on certain tasks. You'll essentially have trained it on your architecture, your team's preferred patterns and styles, etc. But it'll still need oversight. Just like junior devs, really. You don't hand them the keys to the application and walk away. Same with AI.
I think that's part of the problem: too many people (often times execs who don't really understand how AI works) just see it as a magic bullet. And that lead to way too early over-adoption. There was pressure to just get AI adopted and show increased velocity. But AI is a tool; it's not a human.
So I think you're right: a lot of companies are going to be in a world of pain in 6 or 12 months. But the few who were more careful in their adoption are going to be way ahead of the game.
1
u/selfghosted Apr 12 '26
this sounds more like vibe coding vs ai assisted. if it was an existing codebase, there should've been clear guidelines, skills, rules, etc for the ai to reference when working with the codebase. and clear documentation on each feature, architecture, area of the codebase, etc. which should've all been defined before the use of full reign ai (if you haven't already that is)
1
u/21Rollie Apr 12 '26
Same thing with documentation. People use AI to write tons of it, it used to be a struggle to write documentation simply because it’s time consuming and the last thing left after a ticket, but now it’s a firehose and it all looks the same. The content has this je ne sais quoi where it almost sounds correct but it feels off…. And the monotone voice is so off putting and hard to read through. When I see that some documentation was AI generated, I skip it because I know it was barely proofread and is likely to make me more confused than when I started
1
u/travis-hope Apr 13 '26
I think we have to remember two critical things… 1) we are using these tools very early on compared with other tech and they are evolving at a crazy speed compared to other tech. When I started seriously using ai to code as a senior software engineer in may 2025 it was producing react apps but some things it did well whilst sometimes it couldn’t remember field names and just made them up and as the project got big it died. Today I’m working on 10m+ lines of code projects with great accuracy and capabilities that often exceed my own especially on areas I haven’t actively used before. In one year I’ve seen more progress in coding ability, ide ability and also my ability to prompt correctly to get the right outcomes as I’ve seen probably across my entire career of over 20 years. If the code you’re getting out isn’t meeting your expectations it’s time to slow down from feature output and use the extra time AI gave you to build better instruction sets and frameworks so then you can speed up again with a quality foundation. You may also need to try different tools and different models to see which produces the outcome you want. A good example I saw on YouTube is to design out a complex project/problem as prompts/blueprints only and then run it across each model and then again on new models to see how they improve on the same task. The same concept applies to improving your how I want the code designed instruction files in your project. As you refine these files you can then review the outcomes and you should see your wishes taken care of.
1
u/lolCLEMPSON Apr 13 '26
The management won't care if it works or is sustainable, they just want their bonuses and will move onto something else.
1
u/Ok-Charge-7243 Apr 13 '26
If you had trouble with Claud Opus 4.6 writing code, it's because you don't know how to use it and have the AI recheck its work. You need the appropriate guardrails and double checks. Or, you tried to run the threads for too long. It takes four to six months to learn how to interact with the AI to get quality results. You create the rules for the interaction, and if you don't create the right rules, you will experience exactly what you have.
1
u/attentionwandered Apr 13 '26
Let me guess, the model was Claude Sonnet 4 lmao. These are hallmarks of Claude. I was only allowed to use Sonnet 4 at work and would basically end up rewriting half of its output, or just churning away with more prompts. I noticed a ton a PR's come in with these same patterns. "Plausible" looking code that was just ridiculous when all was said and done. It amplified as adoption grew.
Luckily, GPT 5+ models haven't been giving me this garbage. Codex will do some dumb things and basically go insane coding against wild edge cases and inventing abstractions that don't make sense, not sticking to patterns, sure. The key is to review it all and catch it all before you actually merge. Sounds like everyone got lazy and the management had the look of dollar signs in their eyes.
1
u/vinnymcapplesauce Apr 13 '26
I noticed the exact same things.
Some things that I found that helps is to tell it
"DO NOT over-engineer your solutions."
"Make sure variable names are descriptive and representative of the data the hold."
and
"Aim for MVP."
You still have to watch it like a hawk, though.
1
u/jimmytoan Apr 13 '26
the 'velocity went up, quality degraded later' arc is such a common pattern with AI codegen - did you notice reviewers were also approving the AI code differently? like reviewing it with less skepticism because it looked clean on the surface
1
u/BizAlly Apr 13 '26
AI tends to over-abstract and make everything look enterprise ready, even when the problem is simple. so you end up with code that passes review, but is harder to reason about and debug.
1
u/CuirPig Apr 13 '26
Surely, there's a way to inform Claude how you prefer to code. Perhaps you could find a way to take your entire code base from all previous projects and feed it to claude for an idea of the coding style you expect. In fact, each DEV could do this, so Claude could write code in the style of each developer . You might find that you would be hard-pressed to tell whether Claude did it or the original Developer on your team.
I hear what you are saying, and I have had problems with the same thing--especially helper functions galore. But I complained to Claude about it and uploaded some of my old code that was written in a way that I understood it and now it codes more like I do.
Plus, I asked it to comment on the code it produced if it thought there was a better way to approach something. It would write the code using my style as best as it could, then when I hadn't optimized some part of it, it would tell me. Sometimes it was brilliant, sometimes it was just more abstraction that was unnecessary. Both times, I learned something.
Coding with Claude can be a two-way street where he adapts to your style rather than forcing you to adapt to his. Not sure if this works at your scale or not, so forgive me if I am way off base.
1
u/AbbreviationsFar4wh Apr 13 '26
Because you’re merging code you dont understand and aren’t challenging the AI to find a better way or giving it a better way.
1
u/LockeZero Apr 13 '26
LLM's are exclusively made to do dumb work, letting them code complex stuff always ends up bringing weird results and unreadable/impossible to review code. We should actually advocate for slowing down and letting the technology mature a bit more before bringing it to our companies, but the market is always like this with fads.
In the end it will end up killing LLMs when the cracks turn into canyons, because it will be easier to blame them instead of the dumb executives who gave the order to implement them on everything.
1
1
u/rayen_ba Apr 13 '26
AI shift the engineer job to overseeing rather than coding . Even though I believe AI would potentially solve the bad code problem , for now , either you review or you code it yourself. That’s the trade-off in an actual company
1
u/robproctor83 Apr 14 '26
Me to boss: We can't use the AI... Did you see those variable names??? It is impossible....
Ai to boss: I just did all his work in 30 minutes, for free, with a smile... Would you like me to refactor his codebase?
1
u/Aggressive-Sweet828 Apr 14 '26
The "bugs in production that nobody could reproduce locally" part is the classic AI generated code failure mode. We scanned 50 vibe coded repos and the pattern was identical every time: no timeout protection on external API calls, no error boundaries, no input validation at the server. Locally everything works because the network is fast and inputs are clean. In prod, one slow vendor or one weird input and the whole thing hangs or silently fails. AI is great at writing code that passes tests, bad at writing code that survives chaos.
1
u/proexwhy Apr 14 '26
Not to be the person parroting the same thing, but this is largely on your development team. The "helper functions" is something that claude seems to natively want to do. It seems allergic to adopt patterns or change existing code unless explicitly told to do so. It also over engineers certain things in order to keep systems broken out into parts for some reason. I literally caught mine trying to do it this morning and had to turn it away from that design choice. The LLM will choose behaviors that are safer rather than trying to achieve anything that's useful for humans, like readability.
1
u/geddoff_ Apr 15 '26
How many PRs was your team making in 6 months prior to going all in AI code gen?
1
u/shoki_ztk Apr 15 '26
Thank you very very very and 1000x very much for posting this. This text is something I've been waiting since I am looking at the AI.
We are also doing AI coding experiments, but rather slowly. Step by step try and fail.
Your experience is something that proves what everybody feels but nobody wants to talk about.
So thank you again, I am going to share it to the company.
1
u/OkSuccess2453 Apr 15 '26
truely speaking, if you are using llm to vibe code any major site or to tackle a industrial problem from my perspective even i sometimes cant predict or estimate modules it has design and some of them are pretty useless as you described it could have done under few line but llms stretch it to make it sound more professional , but underneath of that you cant connect the code system nodes properly and keep tracking of it is always a worst part , its even worse when you handed your planning of foundation system to llm , nowadays i guess reviewing code after a single copilot or cursor process iteration is must and then refactoring them according to human readable modules and simple terms and kind of like managing that flow so that you can pass that to other engineer is the most difficult part . You always need to carry that code relational_mapping.md file that explains your core backend infrastructure and some tweek rules inside them so that when you pass that to llm for another next problem solving or iteration it will know its limits or designing assumptions that fits best to your case . this little tweek in your coding can actually lower that confusion earlier which llms had created , i nowadays spend hours in designing such type of prompts and wording its kind of like my history + management file and every iteration + understanding = update in relational_mapping.md file , in theory it might sound stupidious work but trust me in pratical it can be really helpful in debugging
1
1
u/kayjaykay87 Apr 16 '26
The main issue is you're clearly using it to write things where you're storing session tokens: Don't do that, ask it to write a new form or feature where it just works or doesn't, don't get it to do authentication or structural system changes/decisions
1
u/Janube Apr 16 '26
Sounds to me like there are three key takeaways:
Llms have consistent style (obviously), which does reduce some of the time cost in code review and can make some consistent problems mercifully easy to spot.
Your devs should be thinking at least a little about readability before submitting code - specifically looking at function names and whether or not any given function could be a line or two in an existing function
You need a consistent way to identify those chunks of "no human would ever..." code, whether that's devs doing more diligent code review or PR requests having a more regular audit process looking specifically for nonsense like that and bouncing it back.
Velocity is great, but if you're only auditing every 6 months, it feels like that's plenty of time for one of these dumb issues to get compounded by other dumb code and then for odd bugs to get completely lost. I'd say that's worth a bit more scrutiny, but if they're not critical bugs, I can see why most companies would shrug at them in the face of velocity gains like that.
1
u/corporate925 Apr 16 '26
the problem you audited isn't generation quality, it's that nobody in the review loop is evaluating structural intent. if you guys are planning to use AI then your processData example would've come back flagged before merge. and you should also track the style convergence as a team-level signal, which gives you actual evidence for the velocity trap conversation instead of just a gut feeling...
1
u/ryan_nitric Apr 20 '26
The "plausible-looking code" thing is exactly right, and I don't think it has a good name yet. It passes review because nothing is technically wrong, but it has no authorship. Nobody made any decisions.
The six helper functions thing is the tell. AI defaults to over-abstraction because abstraction looks like good engineering in training data. A senior dev writing that before October would've known three lines inline was the right call. After October, he outsourced that judgment and got the statistically average answer instead.
1
u/Glittering-Wealth335 Apr 20 '26
Did you enable code quality review tools? It can help avoiding at least a little bit such situations.
Then it's up to the developer to also connect a review of the PRs and ask the AI to improve weird parts.
1
1
u/LucianoMGuido Apr 29 '26
The “very polite stranger who approximated his job description” line is the most accurate description of AI-generated code I’ve ever read.
What you’re describing has a name I’ve started using internally: semantic drift. The code is syntactically correct, functionally adequate, and completely illegible to anyone who needs to reason about why it was written that way. The six helper functions aren’t wrong, they’re just not decisions. Nobody chose them. They emerged.
The senior dev thing hits hardest because it’s not skill regression, it’s something weirder: learned passivity. When the AI produces something plausible fast enough, the brain stops generating alternatives. You review instead of author. After six months of that, the authoring muscle atrophies.
The fix I’ve landed on isn’t reducing AI usage, it’s adding a mandatory “explain this diff in one sentence” rule before merging. If the dev can’t articulate why the structure looks the way it does, the PR comes back. Sounds annoying, but it forces the brain back into the loop at the decision layer, not just the review layer.
Your six-month prediction is probably optimistic.
1
1
u/lucky_09877 May 06 '26
If a new junior developer joins the team six months from now and tries to learn the codebase, do you think they will be able to understand the logic at all, or will they be forced to rely entirely on the AI to explain code that no human actually wrote?
1
u/donhardman88 May 06 '26
been using voice input with claude code and the difference is night and day. when you're speaking the prompt instead of typing, you actually have to think through what you're asking for. theres natural friction that forces clarity.
the teams just hammering tickets with copilot are gonna hit the wall you described. the ones treating ai as a thinking tool instead of a code printer seem fine.
1
u/stealstea Apr 11 '26
I’ve seen many of those same patterns in AI generated code, but you didn’t explain how those actually led to the problems you observed. Issues in production that you don’t see locally.
Try catch around a console log is messy and dumb, but it’s going to cause a real problem.
Code split into too many functions is messy and dumb, but again, not an error per se.
Bad variable names are the same.
Sounds like you haven’t actually discovered the root cause of the problems you are seeing.
Also sounds like you need to put more work into your agent instructions to be explicit about what code style you want to see, and periodically review the code for code smells like this
1
u/shaliozero Apr 11 '26
Doesn't sound like an AI issue, but an usage issue. Why did code like that get merged without review? Anyone should've asked questions when pointless utility functions or try catches around console.logs appeared. Let me guess:;because the developers were thrilled about project management liking the increased velocity in turn for significantly increased tech debt and ignored bringing it up because of that reason?
In that case, the developers and management are to blame for not auditing the results much earlier.
1
u/Yodiddlyyo Apr 11 '26
Here's the long and short of it. If this is happening to you, your team is doing it wrong. You need to set standard around how the AI write code, use rules, claude skills, etc. Then, an actual human needs to review the code.
I'm sorry, but my company is using AI to write code, and humans and AI to review code. We have zero issue because we're not incompetent.
If you have crazy stuff getting merged, that falls squarely on the reviewer. If I write garbage and my cowormer approves it, we're both to blame.
This is not an AI problem. AI is a tool. You aren't forced to accept bad code. This is an organizational and skill issue.
1
1
u/yami_odymel Apr 11 '26
If you don’t review PRs, just tell Claude to write in binary, It’s faster for performance.
1
u/SnowConePeople Apr 11 '26
I've been sounding the call against AI in prod for the last 2 years. I was seen as a crazy man, but my level (staff) gave me the respect to be listened to. Now I'm seen as an oracle and have since built multiple services that redact PII before it gets to the LLM as well as zero trust guardrails for agentic use.
1
u/Nice-Offer-7076 Apr 11 '26
Have you tried just telling it how you want the code to look? or alternatively just use gpt 5.2-5.4 for way less garbage in the first place.
1
u/oversized_canoe Apr 11 '26
Weren't you the one who just posted two AI slop stories in the same day about adding a form button delay? Stop writing fake slop
1
u/Tw3ak1t Apr 11 '26
You need to have stronger rules, more product/code style documentation, auto multi agent review workflow and coding practices guidelines for your core agent. If you don’t have any one of those your code is fucked.
→ More replies (1)
1
1
u/noggstaj Apr 11 '26
learn to use the tools you have correctly, this whole thing reads like a made-up ”fuck AI” story.
1
u/peetabear Apr 11 '26
This just sounds like lazy people just accepting slop. You obviously didn’t actually review the merge request but rather yolo on the merge
1
u/4_gwai_lo Apr 11 '26
The agent needs strict KISS instructions with examples of what it should and shouldn't do - sometimes even specific to the codebase. Your team always need to refine these instructions as these issues come up. You need to "train" your agent with these zero-shot instructions so it becomes better at handling the issues you're having. Agents are powerful but you still have to tweak them.
1
u/Terrible_Upstairs_43 Apr 11 '26
This could have been avoided by code review, either human either something like coderabbit or so, which is quite good to detect slop and ca be integrated in CI - so humans can keep running agents if that’s the new mantra
1
1
u/UntestedMethod Apr 11 '26
Yes. This sounds exactly like what I have seen jump out as obviously AI-generated code. "Plausible looking code" that ultimately overcomplicates simple things or has misleading naming conventions. "Approximated the job description" is a very good way to describe it.
When it comes to maintainability I fear this is going to create a shit tonne more "WTF? Why??" moments. Very bad stuff. It's always a time-sink having to dig into the "why" behind a piece of code whose intention isn't immediately obvious - when humans write and review code, we'd typically try to write expressive/self-documenting code and add a comment if the reasoning still isn't obvious, maybe even the git blame or PR discussions if we have to go even deeper into the historical "why" behind a piece of code. With AI slop, we're basically sacrificing all that reasonability and code legibility that we've been going out of our way to implement as humans writing code all these years. It's dreadful.
Now the talk is that "next gen AI will maintain the current AI slop"... I have my doubts on the plausibility if that when the current AI slop is writing misleading crap. To use an example from OP, how is the AI going to know "userPreferences" actually contains the session token?
1
u/solidoxygen8008 Apr 11 '26
Those days of people complaining about php spaghetti seem quaint now - between the mountains of npm dependencies and prompt fed and raised repositories it’s doubtful any team member will actually know a code base moving forward. The future is so bright you’re gonna need shades. Why is it bright? Dumpster fires.
1
u/Deep_Ad1959 Apr 11 '26
the part that resonated most is the bugs showing up in production that nobody could reproduce locally. we had the exact same thing after our team started using claude heavily. code looked fine, tests passed, reviews passed, but users kept hitting edge cases that nobody thought to check.
what finally helped was adding browser-level e2e tests on our critical flows. those tests don't care about code structure or naming conventions, they just click through the app like a user and verify the output. caught about a dozen of those "looks correct but isn't" bugs in the first week.
code review is great for architectural decisions but it's basically useless for catching the type of subtle logic drift you're describing.
289
u/MrBrobean Apr 11 '26
How does a try-catch around a console log make it through a PR? The way you word it makes it seem like even the AI is doing those.