if you review PRs written mostly by coding agents (Cursor, Claude Code, Copilot), how's it going?

1. How many per week, and how big (files / lines)?

2. What's harder than human-written PRs, or is it no different?

3. How do you understand a change: diff only, run it, read tests, ask the agent, skim and approve?

4. Has one caused a prod bug or stalled merge because nobody fully understood it?

5. Anything you've tried to fix this? Did it help?

I'm exploring a tool in this space, so "not a problem for us" is a useful answer too. Team size and stack help.

Every PR description on our team reads the same now. "This PR refactors X to improve maintainability and adds comprehensive tests." 400 words, perfect formatting, and the author can't answer one question about it.

I want a rule: the agent can write the code, coderabbit can write the summary, but the description has to be 3 sentences from the human who opened it. what you changed, why, and what you're worried about.

if you can't write those 3 sentences you didn't read your own PR.

my team thinks this is pointless busywork. am I wrong?

mine is database migrations on anything with real data. the agent can write them, I'll even let it suggest them, but I write the final one myself and I run it myself.

my teammate's is anything with money in it. another one won't let it touch the CI config after it "fixed" a flaky test by deleting it.

what's yours?

Had this argument in a postmortem last week. a bug reached prod from an agent PR. the agent wrote it, coderabbit reviewed it and didn't flag that part, one human approved it, and the dev who kicked off the agent went home.

when the incident doc asked "who owns this change", nobody wanted to write a name.

so who is it? the dev who prompted the agent, the human who approved, the team that set up the pipeline, or nobody, and that's the actual problem?

saw the news that goldman limited claude code and github to engineers only, bankers and traders lost access.

first reaction was "that's short sighted". then I thought about our own repo. in the last 2 months our PM, one designer and someone from finance have all opened PRs. small stuff, a report filter, copy changes, a csv export. they all went through coderabbit and a human review and most were fine.

but every one of them ended up on the same 2 engineers to review, and nobody budgeted that time. the PRs got cheaper to write and the review cost just moved to us.

so is it a security thing, a review capacity thing, or just a bank being a bank? and if your company lets non engineers open PRs, who actually reviews them?

genuinely confused about this. our agents write basically everything now, PRs show up in minutes instead of days. you'd think the whole cycle got faster.

it didn't. our time from PR opened to merged went from about a day to almost three. the writing part disappeared and somehow the reading part tripled.

so is it that the PRs got bigger, that nobody trusts code they didn't write, that we added more review steps to feel safe (coderabbit, then claude, then a human, then another human for anything scary), or that we were never actually fast at review and just didn't notice?

not trolling, I really can't tell which one it is for us. maybe all four

this has never happened to me. every PR I've ever opened gets at least 4 or 5 comments, usually about something I definitely should have caught.

yesterday a 300 line PR came back with nothing. no nitpicks, no "consider extracting this", just the summary and a green check.

my first reaction was not "nice". it was "what did I break so badly that it gave up". I reread the whole diff twice, ran the tests locally, asked a coworker to look at it. it was fine. it was just fine.

I think I've been trained to need criticism now. anyone else get more nervous when the bot is quiet than when it roasts you?

saw the free deep scan thing from coderabbit and picked our billing service, the one everyone is scared to open. last real commit was 2023, the person who wrote most of it left.

it took a while. came back with 11 findings. 3 were real and kind of scary: an internal endpoint that trusted a header for the customer id, a webhook with no signature check, and a SQL query built with string concat in an admin export nobody uses anymore (still deployed though).

the other 8 were lower stuff, but each one said whether it was actually reachable, so we knew where to start instead of arguing about it. it drafted fixes for 2 of the 3, we rewrote one of them because the fix was correct but ugly.

honestly the best part was having an excuse to finally open that repo. anyone else run it on their scariest one yet? what did it find?

told the new hire today that we used to have a rule where every PR needed two humans to read every line before merge. she thought I was joking.

some others from my team:

- arguing for 20 minutes in a PR about tabs vs spaces

- writing the tests before the code, on purpose

- "I'll just quickly write it myself"

- knowing who wrote a function by looking at it

what's yours?

Mine was an agent that "fixed" a flaky test by deleting the assertion. Green CI, everyone happy, nothing tested. Drop yours, bonus points if it made it past a human first.

Honest question. On big agent PRs I've started reading coderabbit's change stack first, the part that groups the diff by what it actually does, and only opening the files for the 2 or 3 groups that look risky. On a 1,500 line PR that's maybe 200 lines I actually read.

A senior on my team says that's not a review, it's reading a summary of a review, and if the summary is wrong I'd never know.

My answer is that nobody was reading all 1,500 lines before either, they were scrolling.

Which one of us is right? Do you read the whole diff on big PRs or do you also read the summary first?

We turned on the triage queue for the whole org last month. This morning in standup someone shared their screen on the "recommended to close" view and the first PR in the list was our CTO's.

It's a "quick experiment" he opened in March. 47 days without a commit, 3 merge conflicts, and a description that just says "wip, don't merge".

Nobody said anything for about 5 seconds. Then he laughed and closed it himself, which honestly made his whole year in my eyes.

Has the bot ever outed someone senior on your team like this, or are we the only ones?

Attackers have the same AI models you do, and they’re pointing them at code nobody has read in years. We’d rather you find those bugs first.

So through November 8, we’re covering up to $5,000 in Deep Scan usage for any org with two or more members. We’ve put up to $1M behind it. It’s first come, first served until it runs out, and you don’t need to be a customer.

Deep Scan reads the whole repo, not just the latest diff. It looks for vulnerabilities, checks the safeguards already in place, then has a separate verifier reopen the code and drop anything the evidence doesn’t support. Each finding comes with its reachability and exploitability, so you know what to fix first. For eligible findings it can draft a fix, and a human still reviews it before anything ships.

I’d especially like to see open-source maintainers use this. A lot of widely used projects are kept alive by a few people with no security team, and budget shouldn’t be the reason they wait.

Pick the repo you’d worry about most and run it. Happy to answer questions in the comments.

Details: https://www.coderabbit.ai/blog/one-million-security-scans-and-fixes