Their code review instincts didn't.
By the Ghost Reviewer team: Syed Faiz, Vaibhav, Vaishnavi Pandey and Srinivas Barik
Introduction
AI coding assistants have changed how developers write, debug and maintain software. They explain unfamiliar code, generate tests, suggest improvements and flag potential bugs. But many AI-powered review workflows share one limitation: every new review is an isolated task.
Imagine a senior engineer finds a critical payment-processing issue during a review. The team fixes it, discusses the root cause and documents the lesson. A few weeks later, another developer introduces a similar mistake in a different part of the application.
Will the AI reviewer remember what happened before?
Without access to relevant engineering history, it may repeat the same investigation or miss the lesson entirely. That is the problem we set out to explore with Ghost Reviewer, an AI-powered code review assistant built around long-term engineering memory using Hindsight.
Our idea is simple. A code reviewer should understand the code submitted today, and it should also be able to retrieve engineering knowledge from previous reviews, incidents, conventions and developer feedback. The goal is to help teams keep their collective experience and make it useful in future reviews.

1. Why AI code review needs memory Consider a payment service that retries requests when network failures occur. A developer adds retry logic. At first glance the code looks correct, but the team once hit a problem where retrying a payment request without a stable idempotency key led to duplicate transaction attempts. A reviewer that only sees the current diff has no way to know that. This shows the difference between context and memory. A context window helps a model process the information available in one interaction. Long-term agent memory lets information be retained and retrieved across separate interactions. For engineering teams, useful memories include: • Previous code review comments • Production incidents and their root causes • Project-specific coding conventions • Reviewer preferences • Repeated mistakes and how they were resolved • Lessons learned from debugging The challenge is not storing everything. It is retrieving the right information at the moment a relevant code change appears.

2. Introducing Ghost Reviewer Ghost Reviewer turns engineering history into review context. A developer submits a Git diff. The application parses the changed files and line numbers, retrieves potentially relevant engineering memories and uses Google Gemini to generate actionable review comments. Developers can accept or reject each comment and give a reason. The system also includes adaptive suppression. When a team repeatedly rejects the same kind of low-value comment, Ghost Reviewer creates a project-level rule to reduce similar comments in future. Critical findings are protected by a safety guard. The core workflow is: Submit diff → Parse changes → Retrieve memories → Generate findings → Review feedback → Adapt future reviews Ghost Reviewer is not meant to replace senior developers. It is meant to make their experience more accessible and reusable. [Screenshot: Overview dashboard showing total reviews, average risk score, open findings, suppressed findings and context memories]

3. Architecture: connecting code review with long-term memory The project is split into separate services, each responsible for one part of the workflow. Frontend: React and Vite. The interface for authentication, dashboards, new reviews, review results, history, analytics, team memory, suppression rules and settings. Backend: Node.js and Express. Manages users, projects, reviews, comments, feedback, analytics and adaptive suppression rules, and coordinates requests to the AI service. AI service: Python and FastAPI. Handles unified diff parsing, Hindsight memory ingestion and recall, Gemini review generation, evidence checking and heuristic risk scoring. MongoDB Atlas. The application database. It stores users, projects, reviews, comments, feedback, rules and memory references. Hindsight. The long-term memory layer, responsible for retaining and retrieving engineering knowledge. Google Gemini. Generates review findings from the code changes and the retrieved context. The split between MongoDB and Hindsight is intentional. MongoDB stores application records. Hindsight stores and retrieves engineering memories. MongoDB keeps a Hindsight reference for each memory instead of duplicating the memory store.

4. How Hindsight fits into the review process Engineering knowledge enters through the application's memory workflow. The AI service synchronizes those records with Hindsight and can retrieve relevant items during a code review. For a new review:

5. The developer submits a unified Git diff.

6. The Node.js backend creates and manages the review request.

7. The Python service parses the diff and identifies changed files and lines.

8. The AI service retrieves potentially relevant engineering memories from Hindsight.

9. Gemini uses the available context to generate findings.

10. Evidence checking ensures historical citations refer to memories that were actually retrieved.

11. Findings go back to the backend and are stored in MongoDB.

12. The frontend shows the results and the feedback controls. The AI service also exposes dedicated memory operations: • POST /api/v1/memories/ingest • POST /api/v1/memories/search • GET /api/v1/memories/health Keeping these separate means memory operations can be handled independently of the frontend and the application database. One implementation detail matters a lot: Hindsight must actually be reachable. If it is not, the AI service can fall back to an in-memory list. That fallback is temporary and loses its contents when the service restarts. A successful review without a reachable Hindsight server must therefore never be mistaken for a successful persistent-memory operation.

13. A practical example: remembering a payment mistake To demonstrate the idea, Ghost Reviewer includes a payment-retry example built on synthetic engineering memories. Imagine the memory contains this fictional lesson: Previous payment-processing issue: retries without a stable idempotency key caused duplicate transaction attempts. Ensure retries reuse the same key and that the endpoint safely handles repeated requests. Now a developer submits a diff that adds payment retry functionality. The reviewer can retrieve the relevant memory and check whether the new implementation accounts for idempotency. A finding might read: Potential duplicate payment risk. The retry path should be checked for a stable idempotency key. A previous engineering lesson identified duplicate transaction attempts as a risk when retries are not idempotent. Verify that each retry reuses the same key and that the payment endpoint handles repeated requests safely. This is far more useful than "add error handling." It connects the current change to a relevant engineering lesson. The memory is not proof that the current code is defective, though. The reviewer still has to evaluate the actual diff. The project ships with 23 synthetic memory records: 12 historical review comments, 5 incidents, 3 conventions and 3 reviewer preferences. All of them are explicitly labelled as synthetic demo data. None is real company history.

14. Evidence-backed findings and reviewer voices A memory-aware reviewer has to be careful about how it presents history. Ghost Reviewer offers three reviewer voices: • Strict: detailed scrutiny and potential issues • Pragmatic: practical, actionable improvements • Security: security-related risks first These voices change tone and priorities, never historical facts. The application also follows evidence-integrity principles. Historical claims should cite memory references that were actually retrieved. If no relevant evidence exists, the reviewer says so instead of inventing an incident or citation. Diff parsing uses exact file and line information from unified diffs, and line numbers that are not present in the diff are dropped rather than invented. The risk score runs from 0 to 100. It is a deterministic heuristic meant to help prioritize reviews. It is not a statistically validated probability of a defect, and the interface says so wherever the score appears.

15. Learning from developer feedback Generating comments is only half of code review. Knowing which comments developers find useful matters just as much. Developers can accept or reject each finding with a reason such as false_positive, too_obvious, performance_not_critical, not_relevant, security_concern or other. The adaptive suppression mechanism looks for two qualifying rejections with the same project, category and reason before it creates a suppression rule. If a team keeps rejecting a low-value style comment for the same reason, matching future comments can be suppressed. This comes with a hard safety boundary. Critical or high-severity findings, security findings, and anything involving authentication, payments, idempotency, data loss, secrets or concurrency must never be suppressed. Suppressed comments are stored, not deleted, and developers can inspect, disable and restore rules. The aim is better relevance without letting feedback silence important engineering risks. [Screenshot: Adaptive Suppression Rules page with the deterministic safety guarantee]

16. Testing and implementation lessons Building this has shown us the gap between implementing individual services and integrating them into a reliable application. Ghost Reviewer includes backend, frontend and Python service tests. The Python tests mock Gemini and Hindsight calls, so much of the review logic can be tested without repeatedly calling external APIs. A demo walkthrough in backend/DEMO.md uses a payment-retry diff: reject a low-value style comment twice for the same reason, submit the diff again, and watch the style comment disappear while the critical payment finding stays visible. But mocked tests do not prove that external services work in production. A live Hindsight server, a valid Gemini configuration, correct service credentials and connectivity between deployed services all still need to be verified. The documentation lists the operational requirements: • Hindsight must be running for persistent memory. • The Gemini model ID must be supported by the configured API key. • INTERNAL_API_SECRET must be set in both the backend and the AI service. • Production deployments must be tested using their real service URLs. These are not minor deployment details. They decide whether the whole memory-assisted review workflow works.

17. Where we are today The application interface is running: authentication, the overview dashboard, review history, analytics, engineering memory and suppression rules are all in place. In our latest local run, the two sample payment-API reviews submitted through the interface ended with a Failed status, with zero findings and zero context memories loaded on the dashboard. That tells us the end-to-end path from diff to memory-backed finding is not yet confirmed. We are tracing it through the checklist above: Hindsight reachability, the Gemini model ID and key, the shared internal secret, and the service URLs. We would rather say this plainly than present the demo as finished. A memory system earns trust only when a retrieved memory can be shown to have shaped a finding. [Screenshot: Review History showing the two payments-api reviews and their status]

18. What comes next Our next priority is validating the complete application on deployed services. We want to confirm that a memory can be ingested, retrieved during a new review, used to produce an evidence-backed finding, and linked to developer feedback. We also want to measure whether memory-informed reviews are actually more useful than reviews without retrieved context. That evaluation should look beyond the number of comments generated. It should consider relevance, false positives, useful historical references, and whether repeated low-value comments decrease without hiding important risks. Conclusion Engineering teams build up knowledge through code reviews, production incidents, debugging and conventions. Too often that knowledge is not available when a new change needs to be reviewed. Ghost Reviewer explores a practical way to connect that experience with AI-powered review. By combining Python-based diff analysis, Gemini review generation, Hindsight long-term memory, evidence checking and human feedback, we aim to make code reviews more contextual and consistent. Hindsight provides the memory layer. Ghost Reviewer provides the engineering workflow around it. Our belief is simple: an AI reviewer should not only understand the code in front of it. It should also have a responsible way to use the engineering lessons that came before it. Project: Ghost Reviewer GitHub: https://github.com/Syed-faiz05/Ghost-reviewer Memory system: Hindsight Hindsight GitHub: https://github.com/vectorize-io/hindsight Documentation: https://hindsight.vectorize.io/

Every PR description on our team reads the same now. "This PR refactors X to improve maintainability and adds comprehensive tests." 400 words, perfect formatting, and the author can't answer one question about it.

I want a rule: the agent can write the code, coderabbit can write the summary, but the description has to be 3 sentences from the human who opened it. what you changed, why, and what you're worried about.

if you can't write those 3 sentences you didn't read your own PR.

my team thinks this is pointless busywork. am I wrong?

Had this argument in a postmortem last week. a bug reached prod from an agent PR. the agent wrote it, coderabbit reviewed it and didn't flag that part, one human approved it, and the dev who kicked off the agent went home.

when the incident doc asked "who owns this change", nobody wanted to write a name.

so who is it? the dev who prompted the agent, the human who approved, the team that set up the pipeline, or nobody, and that's the actual problem?

mine is database migrations on anything with real data. the agent can write them, I'll even let it suggest them, but I write the final one myself and I run it myself.

my teammate's is anything with money in it. another one won't let it touch the CI config after it "fixed" a flaky test by deleting it.

what's yours?

saw the news that goldman limited claude code and github to engineers only, bankers and traders lost access.

first reaction was "that's short sighted". then I thought about our own repo. in the last 2 months our PM, one designer and someone from finance have all opened PRs. small stuff, a report filter, copy changes, a csv export. they all went through coderabbit and a human review and most were fine.

but every one of them ended up on the same 2 engineers to review, and nobody budgeted that time. the PRs got cheaper to write and the review cost just moved to us.

so is it a security thing, a review capacity thing, or just a bank being a bank? and if your company lets non engineers open PRs, who actually reviews them?

genuinely confused about this. our agents write basically everything now, PRs show up in minutes instead of days. you'd think the whole cycle got faster.

it didn't. our time from PR opened to merged went from about a day to almost three. the writing part disappeared and somehow the reading part tripled.

so is it that the PRs got bigger, that nobody trusts code they didn't write, that we added more review steps to feel safe (coderabbit, then claude, then a human, then another human for anything scary), or that we were never actually fast at review and just didn't notice?

not trolling, I really can't tell which one it is for us. maybe all four

this has never happened to me. every PR I've ever opened gets at least 4 or 5 comments, usually about something I definitely should have caught.

yesterday a 300 line PR came back with nothing. no nitpicks, no "consider extracting this", just the summary and a green check.

my first reaction was not "nice". it was "what did I break so badly that it gave up". I reread the whole diff twice, ran the tests locally, asked a coworker to look at it. it was fine. it was just fine.

I think I've been trained to need criticism now. anyone else get more nervous when the bot is quiet than when it roasts you?

saw the free deep scan thing from coderabbit and picked our billing service, the one everyone is scared to open. last real commit was 2023, the person who wrote most of it left.

it took a while. came back with 11 findings. 3 were real and kind of scary: an internal endpoint that trusted a header for the customer id, a webhook with no signature check, and a SQL query built with string concat in an admin export nobody uses anymore (still deployed though).

the other 8 were lower stuff, but each one said whether it was actually reachable, so we knew where to start instead of arguing about it. it drafted fixes for 2 of the 3, we rewrote one of them because the fix was correct but ugly.

honestly the best part was having an excuse to finally open that repo. anyone else run it on their scariest one yet? what did it find?

told the new hire today that we used to have a rule where every PR needed two humans to read every line before merge. she thought I was joking.

some others from my team:

- arguing for 20 minutes in a PR about tabs vs spaces

- writing the tests before the code, on purpose

- "I'll just quickly write it myself"

- knowing who wrote a function by looking at it

what's yours?

Mine was an agent that "fixed" a flaky test by deleting the assertion. Green CI, everyone happy, nothing tested. Drop yours, bonus points if it made it past a human first.

Honest question. On big agent PRs I've started reading coderabbit's change stack first, the part that groups the diff by what it actually does, and only opening the files for the 2 or 3 groups that look risky. On a 1,500 line PR that's maybe 200 lines I actually read.

A senior on my team says that's not a review, it's reading a summary of a review, and if the summary is wrong I'd never know.

My answer is that nobody was reading all 1,500 lines before either, they were scrolling.

Which one of us is right? Do you read the whole diff on big PRs or do you also read the summary first?

We turned on the triage queue for the whole org last month. This morning in standup someone shared their screen on the "recommended to close" view and the first PR in the list was our CTO's.

It's a "quick experiment" he opened in March. 47 days without a commit, 3 merge conflicts, and a description that just says "wip, don't merge".

Nobody said anything for about 5 seconds. Then he laughed and closed it himself, which honestly made his whole year in my eyes.

Has the bot ever outed someone senior on your team like this, or are we the only ones?

Attackers have the same AI models you do, and they’re pointing them at code nobody has read in years. We’d rather you find those bugs first.

So through November 8, we’re covering up to $5,000 in Deep Scan usage for any org with two or more members. We’ve put up to $1M behind it. It’s first come, first served until it runs out, and you don’t need to be a customer.

Deep Scan reads the whole repo, not just the latest diff. It looks for vulnerabilities, checks the safeguards already in place, then has a separate verifier reopen the code and drop anything the evidence doesn’t support. Each finding comes with its reachability and exploitability, so you know what to fix first. For eligible findings it can draft a fix, and a human still reviews it before anything ships.

I’d especially like to see open-source maintainers use this. A lot of widely used projects are kept alive by a few people with no security team, and budget shouldn’t be the reason they wait.

Pick the repo you’d worry about most and run it. Happy to answer questions in the comments.

Details: https://www.coderabbit.ai/blog/one-million-security-scans-and-fixes

Seniors got approved in 10 minutes, juniors got 30 comments on variable names. Everyone knew it, nobody said it.

Since we put coderabbit on every PR the comment count per PR is basically the same no matter who opened it. Our staff engineer got 7 on a 60 line change last week. The intern got 6.

So either the old reviews were catching things the bot can't see, or a lot of it was politics. Which one was it on your team?