Two Reviewers, Every Pull Request

Five years ago we were twelve people with an office in Kharkiv. Today we are six, we serve more clients than we did then, and we just lost our senior code reviewer on short notice. Instead of hiring, we put two independent AI reviewers on every pull request. Thirty days and 377 pull requests later, here is what the data says, what it costs, and where it falls short.

Three weeks ago our most experienced reviewer became unavailable overnight. He was the person who read every pull request before it merged. There was no one to absorb that role, because there has not been slack on this team for a long time.

Five years ago we were twelve, with an office in Kharkiv. Then the war came. People relocated, life intervened, and the building is no longer ours. The team did not shrink in one cut. It shrank slowly, to half, and each time someone left, the work they did had to go somewhere. It went to automation. Deployments, monitoring, provisioning, content builds, and most of the code itself are now done by systems we wrote or rent. The client list did not shrink. We serve more clients today than we did at twelve.

Code review was the last step that still ran entirely on one person's attention. So when that person was gone, we did what we had done every other time. We changed the rule. Every pull request in our GitHub organization now gets reviewed by two AI systems that share nothing with each other: GitHub Copilot code review and Devin Review. A human still presses merge. But the first read, the line-by-line one, is no longer a person's job.

I pulled the numbers for the last 30 days to see whether this is working or whether we are fooling ourselves.

Thirty days of pull requests

The line that matters is the second to last. Roughly one in three merged pull requests got at least one more commit after a bot left a comment. The reviewers are not decorative. They change what ships.

The other line worth staring at is the one that is not in the table. Human reviewers left comments that led to a follow-up commit on six pull requests. Six, out of 377. Most human involvement was an approval with no comment. That was already true before we lost our reviewer. It was just less visible.

Why two, and why they cannot talk to each other

I write most of our code with Claude. If I then asked Claude to review that code, I would be asking the same model, with the same blind spots, to grade its own homework. The point of a reviewer is a different perspective.

So the rule is two different vendors, two different model stacks, and no shared context. Copilot sees the diff and posts inline comments. Devin clones the repository, explores the surrounding code, and posts its own findings. Neither one reads the other's review. When they both flag the same line, that is a strong signal. When only one does, a person looks.

This is the same instinct behind separation of duties in traditional change control. The author should not be the sole approver. We have just widened the definition of "someone else" to include a second machine that did not write the change.

What the bots actually catch

I expected Copilot to be the shallow one. It is not. Its inline comments in the window are mostly cross-file: a release guide that says schema changes require a minor version bump while the pull request ships a patch; a workflow that builds from an env file the pull request just emptied; a Terraform plan that reports a destroy the pull request title never mentioned; a mobile breakpoint rule that only targets one heading level after a page promoted its title to another. It also catches the classic small bugs, like comparing timestamp strings lexically or a CSV parser that eats escaped quotes. And it is chatty. On the pull requests I sampled, Copilot opened about four inline threads for every one Devin opened.

Devin is quieter and reads more of the repository. On most pull requests it reports nothing at all. When it does speak, each finding carries a severity mark and, for the ones it considers fixable, a ready-to-paste prompt for a coding agent. Its best comments in the window were about certainty: a forensic CLI that could declare a spreadsheet cell "never changed" when a revert inside one revision made that unprovable; a pagination call that silently dropped everyone after the first page; a lazy-loading script with no fallback for browsers that lack the observer API it depends on. Those are design flaws, not typos, and a diff-only reviewer would not see them.

Sometimes they converge. Both reviewers independently flagged the same responsive CSS rule on the same line in the same pull request. That is the moment the two-reviewer design pays for itself, because two unrelated systems agreeing is far more convincing than either one alone.

The 101 follow-up commits are mostly small: a guard clause, a corrected environment variable, a fallback branch, a doc brought back in line with the code. None was dramatic. All of them would have been a production surprise or a confused teammate a week later.

Of the 608 inline threads, 424 were resolved. The rest were noise, dismissed on purpose, or attached to pull requests that never merged. We closed 54 pull requests without merging in the window, and in some of those the reviewers did their job by making it obvious the change was not ready.

What it costs

Copilot bills code review in AI credits. In our billing export, the repo-attributed credits for the window came to about $25, which works out to roughly four cents per review pass. GitHub's own documentation puts a Lite review at $0.05 to $1 in credits and a Balanced review at $0.25 to $5, so we are at the cheap end. Seats are the real line item: Copilot Business plus GitHub Code Quality licenses run about $160 a month for the team.

Devin Review does not expose per-review cost through its API, so I cannot give you a clean per-pull-request number. All in, with both tools and their seats, the setup is around $650 a month for 300-plus pull requests.

I also estimated the token volume from the diff sizes. Both reviewers together read on the order of 15 million input tokens and wrote about a million output tokens in the month. At list prices for frontier models that is around $60 of raw inference. The vendors are charging for the workflow, not the tokens.

Compare that to the salary of the person we lost, or to the cost of the three hires we would have needed over five years to keep the old headcount. Token cost is not the constraint. It never was. The constraint is how much of the work can be handed to something that does not leave.

A different design: reviewers on every laptop

On a weekly call with a few founder friends, one team walked through the reviewer they built for themselves, and it is a genuinely different architecture worth describing.

Their trigger is a label on the pull request. A bash script on each developer's laptop polls GitHub every 60 seconds for labeled pull requests authored by that developer. When it finds one, it launches a set of Claude Code sub-agents, each with its own isolated context and a specific job: one for correctness, one for security, one for style, and one custom check that matters to them more than any of the others.

That custom check is about data ownership. A change can be correct, tested, and clean, and still read a value from a cache that expires in a week, or from a table that belongs to a different service. No generic reviewer catches that because no generic reviewer knows the team's domain boundaries. Theirs does, because they wrote the rule down as a skill and gave it to a dedicated agent.

Two design decisions stood out. First, the supervisor is a bash script, not an agent. Orchestration logic in plain code costs zero tokens, and they found it did not reduce quality. Second, each developer runs their own instance for their own pull requests, so the cost lands on the subscription of the person who opened the change. They estimated the review would cost around $7 per pull request at raw token prices, but on a flat subscription it is effectively free.

Their bot can approve. If it approves, a second agent merges. If it leaves comments, a third agent pushes fixes and re-labels the pull request for another pass. Their engineering lead said AI now writes about 90 percent of their code and will soon review 100 percent of it.

I asked why they did not just use Copilot. The answer was that the generic tools are shallow. They find the obvious bug. They cannot find the bug that only exists because of how that team's data is organized. If your domain rules are the thing that matters, you will end up writing the reviewer yourself.

Our approach and theirs are not in conflict. We rent two general reviewers. They built one specialist. Both replace the first human read. The right choice depends on whether your risk lives in the code or in the domain, and on how many people you have to maintain the thing. They are a growing team with the headcount to own a custom reviewer. We are six people who need every tool to work the day it is installed and keep working when nobody is looking at it. Renting was the right call for us.

The compliance question

We are SOC 2 compliant, and the obvious objection is that separation of duties requires a second person, not a second model. At twelve people that was easy to staff. At six, with everyone already carrying two roles, a second human on every change is the kind of control that looks good in a policy and quietly stops happening in practice.

It turns out the framework does not say that. The change management criterion, CC8.1, asks that changes be authorized, tested, and approved through a process you wrote down. It does not name who or what performs the approval. A recent analysis by heygrc walks through this carefully: automated approval for a defined class of change can satisfy the criterion, if the control is scoped, evidenced, and operates the way the policy describes.

The hard part is not the rule. It is the evidence. You have to be able to show an auditor which pull requests were reviewed, by what, with what result, and that a human still approved the merge. We keep that trail in GitHub, and I am building a collector that pulls it into a compliance index so the evidence exists before anyone asks for it.

If you go this route, write the policy before you change the practice. The tooling is the easy half.

What the research says, and why I am still cautious

The evidence on AI code review is mixed, and it would be dishonest to leave that out.

A study accepted at MSR 2026 looked at 3,109 pull requests and found that pull requests reviewed only by code review agents merged at 45 percent, versus 68 percent for human-only reviews. Twelve of the thirteen agents they measured had average signal ratios below 60 percent. Their conclusion was that agents should augment human reviewers, not replace them.

LinearB's 2026 benchmarks report, built from 8 million pull requests, found AI-authored pull requests merge within 30 days about a third of the time, against 84 percent for human-authored ones.

Our numbers do not contradict that. Our merge rate in the window was 76 percent, which is high, but our pull requests are small, our reviewers are two rather than one, and a human still decides. The research is measuring what happens when you remove the human entirely. I am not doing that. I am moving the human from first reader to final approver.

The other honest caveat is that a third of pull requests getting a follow-up commit tells you the reviewers found something. It does not tell you what they missed. I do not have a number for that, and neither does anyone else who publishes these stats.

What I would tell a team of six

A few things I would do again, and a few I would do sooner. Most of them are less about AI than about running a small team that keeps getting smaller and cannot let the output shrink with it.

Use two reviewers from two vendors, and never let them share context. A single model reviewing its own output is theater.

Keep the human on the merge button. Not because the framework requires it, but because the merge is where the accountability lives.

Read the review comments for the first month. The friend's team did exactly this and it is why their custom check exists. You cannot tune a reviewer you do not read.

Instrument it. The GitHub GraphQL API gives you reviews, threads, commits, and timestamps for every pull request. One script and you know whether the bots are changing what ships or just adding comments.

Keep a second coding model configured and ready. The day one model hits its usage cap, or balks at a legitimate security patch on your own infrastructure, is never a slow day.

And write the policy down. The rule "two independent reviews, one human merge" is easy to state and easy to evidence. It is the kind of control an auditor can test. It is also, as it turns out, cheaper and more consistent than the thing it replaced.

We did not choose this. We were pushed into it, the same way we were pushed into every other piece of automation over the last five years. Thirty days in, I would not go back. Six people with two tireless reviewers ship more than twelve people with one tired one ever did.