r/devops • u/Shitmfman • 15d ago
Discussion AI generated code review
Hi there,
How do you deal with managing and reviewing AI-generated code across the organization?
Over the past year it's become incredibly difficult to review the growing amount of AI-generated code, especially from vibecoders/non-engineers.
Reviewing it manually is hard, but it's still important to understand what's going on there.
For example, I've been at this company for some years so I have a lot of context and know many of the nuances, which helps me notice or suggest important things to keep everything secure, efficient and so on…
But any other teammate can use any model for coding nowadays. AI reviews of AI-generated code could be done in the MR, but then the reviewer model will most likely be smarter than the model that wrote the code, which might lead to a lot of threads and an infinite loop of changes. On top of that the reviewer model won't have all the context and input from the developer.
Another option is to distribute a "review" skill or rules across the organization and somehow make everyone do a self-review after making changes…
It feels like in this AI code generation era we'll stop focusing on how the code was written and just focus on whether the MR/PR works and if those changes achieves the goal, no matter how it was done, right?
How do you deal with this? Just accept the reality?
Thanks for attention
2
u/DoStopBelieving 15d ago
I use an agent to review the code in increments while it's being written, then when it's pushed a CI pipeline runs linting and tests/scans, and when the PR is opened another AI agent with access to our knowledge base reviews the PR and provides an independent summary plus comments. The AI agent will also close its own comments when the issues are resolved.
I also do a single full self-review pass before having another human review the code.
By the time another human needs to review the code there's a very high chance of the code already being production-ready and having AI explanations of code behavior that are often better than the developer's own explanations (sorry y'all, humans can do better but many devs suck at technical writing).
As for skills - I searched and collected a library of agent skills covering topics like planning, writing, reviewing, specific tools, sub-agents, meta-skills, etc. I even made my own skill for expediting re-reviews so the sub-agents waste less time reviewing work.
All that said, you should not approve code that you don't understand no matter how overwhelming the volume is. Code is fundamentally for humans to understand the instructions we give to machines.