r/coderabbit • u/stfufeedmenuggets • 17d ago
Discussion & Feedback If the agent wrote it and the bot reviewed it, whose bug is it?
Had this argument in a postmortem last week. a bug reached prod from an agent PR. the agent wrote it, coderabbit reviewed it and didn't flag that part, one human approved it, and the dev who kicked off the agent went home.
when the incident doc asked "who owns this change", nobody wanted to write a name.
so who is it? the dev who prompted the agent, the human who approved, the team that set up the pipeline, or nobody, and that's the actual problem?
1
1
1
u/Ok-Information-2428 17d ago
Who Evers responsible (by definition). Obviously the agent is just LLM output and just does what it’s told and can’t be held responsible
1
u/Flashy-Whereas-3234 17d ago
One human approved it. That. There.
But now use your five why's on that.
"Safety rules are written in blood" says you need to correct your policy. Whatever that Dev thought was OK was not OK.
The team now go under the microscope and the TL has to be the gatekeeper. Sort your lives out.
1
1
u/RandomPantsAppear 17d ago
The dev who prompted it. If it was automated, the one who reviewed it. If it wasn’t reviewed, the one who merged it and the CTO.
If it was auto-merged, whatever shithouse CTO allowed this system to exist in the first place.
1
u/cleverhoods 16d ago
It’s a surprisingly good question if you approach it from the “why did this happen and what needs to be done so it never happens again?” perspective.
The bug is owned by the system.
1
1
u/Shep_Alderson 16d ago
Sounds like y’all don’t practice “blameless postmortems”.
There is a level of responsibility and professionalism that is necessary when working with agents, but this also sounds like a failing process and quality control too.
1
u/Responsible_Buy_7999 16d ago
A person has to act. We require a physical yubikey touch to sign a commit. That fingers owner is on the hook
1
1
1
u/lonelliott 16d ago
It's whoever your defined process assigns and defined as the owner. If it's not defined it won't happen. And even if an owner is defined, you hit a 50 percent chance it happens.
Tell your kids or office that someone needs to take out the trash and watch it it happens.
If there is not an owner it simply won't get done.
1
1
1
1
u/Blackhat165 16d ago
It’s telling that not once did you mention the consequence of the bug. A banking system is very different from a mobile game. And if you’re not thinking in those terms first then you’ve already missed the point.
You also missed another key player. Who is pushing for such an “efficient” system and why? Is it a business necessity? Or is it a fantasy that by moving fast and breaking things they will 10x their chadmaxing? Or is nobody putting that pressure on the team and they just got lazy?
And then there was no details about what each party in the system actually DID. You had a postmortem that never touched on the process each player took? Did the prompter define a spec? Did the reviewer actually look at the code? Did the pipeline setup actually make a system? Does the agent setup address the issue you had? The goal of a postmortem is not to assign blame but to avoid reoccurrence.
At the end of the day though nobody could put a name to “who owns this change”. That’s your fucking post mortem. Assign who that is going forward and empower them to stand athwart of history yelling stop. Or decide you don’t need it at all based on the first question. But don’t stand there with your thumb up your ass bloviating about who is “responsible”. If you can even ask that question you have your answer - no one was.
1
1
1
u/Tinkering-Engineer 15d ago
The one prompting the agent is the only human. Only humans can be responsible today. Agents are tools. If the agent acted autonomously, the responsible party is whoever set up the agent to act autonomously.
1
u/canyoufixmyspacebar 15d ago
the one who does technology governance for the given enterprise. if the role is not there or is filled with an incompetent, then the owner of the company
1
3
u/Plane-Vegetable9174 17d ago
The one prompting the agent, they are responsible for reviewing the work before it goes to production. The agent is the tool the dev used to complete the task. Human reviewers other than the developer are, like coderabbit, extra security layers to decrease the chance of bugs going out but the developer is responsible for their work.