When Claude writes ~90% of a codebase and merged changes multiply, line-by-line human review stops scaling. How the Claude Code team moved the reading to machines, what each layer is actually for, and which of their numbers you should not copy.
Spine text (claude.dev): How the Claude Code team does code review · Supporting: Code Review docs · Redesigning source control
The spine post is written by Anthropic about Anthropic’s own repos and a product it sells. It is unusually candid (it publishes a failure, a 50-round review loop, and admits missing circuit breakers) but its headline results are internal, unaudited and have no counterfactual. I separate what the source says, what it leaves unmeasured or self-serving, and independent or primary evidence (the product docs and Anthropic’s source-control post). Arithmetic is mine and marked.
Modules 1 and 5 make you commit a number before the reveal; modules 3 and 6 make you commit a decision. Related courses in this library: The Harness Is the Policy (gates and merge queues), The Attention Factory (OpenAI’s pipeline) and Harnesses That Fit the Task (why same-model verifiers add independence of context, not of error).
The post’s opening facts: Claude writes about 90% of Claude Code’s own code; between June and September merged PRs more than doubled while the share needing a revert fell by over 40%. Separately, Anthropic’s monorepo went from roughly 2,500 changes landing per day in May to more than 30,000 in September (per Redesigning source control), about a 12× rise (my arithmetic). Claude Code lives in its own repo but feels the same pressure.
The team did not remove review. It moved most of the reading to machines, spent human attention where machines are weak, and made mistakes cheap to undo. The post separates review’s two jobs: (1) find and fix issues before they ship, and (2) build shared understanding of the codebase. It optimizes the first and substitutes something else for the second (module 7).
The target is that most PRs across the CLI, desktop/web and mobile repos merge with at most one human in the loop (the author counts), with no increase in reverts or incidents. The metrics are human approvals per PR (median and p95), human comments, and time a PR waits on a person. Revert and incident rates are guardrails: if they rise, human time was moved, not saved.
Suppose every one of 30,000 changes per day needed one human to read it for 5 minutes. How many engineer-hours per day is that? Commit a number, then see what it implies.
“PRs doubled, reverts fell 40%” is a correlation across a window in which almost everything else also changed (new models, flags, staged rollouts, more automated checks). The post gives no counts, no incident denominators and no comparison repo that kept human review. Treat it as a plausible existence proof, not a measured effect of the process described. Also note the revert share can fall simply because the denominator (merged PRs) doubled, so ask for absolute reverts and their severity.
Review has a third job the post underweights. Microsoft researchers (Bacchelli & Bird, ICSE 2013) found that developers expect review to find defects but that its realized outcomes skew toward knowledge transfer and team awareness. If you remove reading, you remove that channel too; the post’s answer is explainers (module 7), which is untested.
| Stage | What runs | Failure class it targets |
|---|---|---|
| Before the PR | Claude runs tests, build and linter and shows the output; a subagent reviews the diff in fresh context; internal skills for bug hunting, diff simplification and end-to-end behavior checks; UI checked against design guidelines | Cheap mistakes, unverified claims of “done” |
| Every PR | Required CI (lint, types, cross-platform tests, build, packaging); AI code review; separate security review; risk-based human approval | Regressions, logic bugs, vulnerabilities, judgment calls |
| Merge | Merge queue re-runs checks against the latest main with the change applied | Two individually green PRs that break main together |
| After merge | Features behind flags, internal-first; releases to latest before stable; author and on-call watch, then fix forward or roll back | Whatever everything above missed |
The structure is defence in depth, and the load-bearing assumption is in the last row: everything is reversible and blast radius is small. Review quality matters less if a flag flips off without a deploy. This is why the same post that cuts human reading also insists on flags, staged releases and one-step rollback.
Boris Cherny (Claude Code’s creator) is quoted that verification is probably the most important thing for good results, estimated at two to three times better. The post’s checks before the PR are an application of that: the agent must show runnable evidence, not assert it.
In Redesigning source control, the monorepo team says a global repository lock for a merge queue was not practical at 30,000 changes a day, so they invested in post-merge and pre-deploy testing instead and rely on Claude fixing conflicts quickly. The Claude Code repo uses a queue; the monorepo does not. A merge queue is a throughput-limited safety device. Know your volume before adopting it as a requirement.
The layers are described but their marginal catch rates are not. Which layer found the bugs that mattered? Without per-layer attribution you cannot tell which to cut when cost or latency bites. The post’s incident reviews ask “which layer should have caught it”; publish that table internally and you have the data to prune. I would also keep one layer deterministic and non-AI as the floor: required tests, types and path-based ownership rules should hold even if every model in the pipeline is down or wrong.
| Risk | Examples in the post | Human approvals |
|---|---|---|
| Low | Docs, copy, internal-only features, additive changes behind a flag | None, once automated and security checks are green |
| Medium | Ordinary product changes; sensitive areas such as caching, the auto-updater, system prompts | One person |
| High | Highest-risk code; a small set of paths that always need two | Two people |
An LLM tier classifier is itself an unreviewed component with write authority over your review policy. The post hedges well (a second classifier can only escalate, and path lists and CI files are hard rules), but you should make the asymmetry explicit: deterministic path rules set a floor, a model may raise the tier, and nothing model-derived may lower it below the floor. If a prompt-injected PR description can talk the classifier down to “low,” your whole system is a text field.
“The prompting person counts as a reviewer” is a weak control. They wrote the intent, saw the agent’s summary and are the most likely to rubber-stamp it (automation bias). It satisfies accountability, not independent verification. Count it for the attention it adds, not the independence it lacks.
REVIEW.md versus CLAUDE.mdThe reviewer reads the diff against the whole repo and the PR’s stated intent; for bug fixes it looks for the root cause. The product docs describe four stages: several agents find candidates in parallel; a verification step checks them against the code and drops false positives; survivors are deduplicated and ranked by severity; the review posts inline comments, a summary and a check run. The post adds that a separate group votes on each finding and only findings with enough agreement are posted, and that style, diagnostic and cleanup-only findings are dropped, with the filters A/B tested on internal PRs. Admins choose Quick, Standard or Deep per repo; Deep is for places where one missed bug costs more than the review time.
Comments are tagged Important, Nit or Pre-existing, so inherited bugs are not blamed on the author. When the research preview launched, the share of internal PRs getting substantive comments rose from 16% to 54% (about 3.4×, my arithmetic); most PRs over 1,000 lines get findings and most small ones come back clean.
CLAUDE.md describes the project: stack, conventions, goals and non-goals. It is loaded on every review, so keep it short and link out. The reviewer also flags convention violations and stale lines.REVIEW.md is written by code owners to say what matters in their area. Good ones are short, describe what failure looks like in that repo (docs that “lie,” config that “misroutes”), cap nits, list paths to skip, require file-and-line evidence and start summaries with a count. The docs add a security-relevant detail: it is read from the base or default branch, never from the PR’s own changes, so a PR cannot rewrite its own reviewer.When the reviewer keeps flagging the same thing, the team fixes it at the source: the skill, a lint rule or an instruction file. That is the right instinct: a repeated review comment is a missing automated check.
Per the docs, a review works from reading the diff and surrounding code, without building or running your code, with read-only repository access, and a clean review “doesn’t mean your code is error-free or secure.” Behaviour bugs the reader cannot infer (timing, platform quirks, integration) must still be caught by tests, which is why the post keeps CI and pre-PR execution in front of it.
Voting among agents from one model family is weaker evidence than it looks. The post says findings need enough agreement; but if the agents share training and blind spots, agreement filters noise (random hallucinations) far better than it filters shared misconceptions. It will raise precision and can lower recall on exactly the subtle bugs where models are all wrong the same way. The post A/B tested the filters, but on what outcome? It does not say whether recall was measured. Insist on seeing both.
“Open findings block merge” is your configuration, not the product’s behaviour. The docs say a review never approves or blocks and the check run does not conclude as a failure. The post’s blocking comes from requiring review threads to be resolved (fix, or close with a one-line reason). That is a sensible mechanism, but if you adopt the product without enabling it, nothing stops a merge. Also note it creates a pressure to resolve-with-a-reason; track how often reasons are “intentional” versus “wrong.”
The 16%→54% figure measures comments, not correctness. More substantive comments is only good if they are right. The post does measure author response elsewhere (module 6), but this headline does not use it.
REVIEW.md from the base branch, short and failure-shaped, caps nits; repeated comments become lint rules or skills.One CLI PR went through more than 50 review-and-fix rounds overnight between agents. Git blame showed about 59% of what the reviewer kept flagging was old code present since its first pass. Each round found a few more old bugs and rarely new ones. The reviewer was reading the whole surrounding area each time and always finding something, so the PR could never reach “clean.” Two honest observations: the finds were mostly real old bugs (a feature), but they were being charged to a PR that did not introduce them (a process bug).
| Internal PRs | Before | After |
|---|---|---|
| Passing after one review-and-fix round | 65% | 78% |
| Needing five or more rounds | 8.9% | 1.5% |
| Mean reviews to pass | 2.08 | 1.32 |
Using the mean-reviews figures above, across 1,000 PRs, how many reviewer runs did the convergence changes save? Commit a number, then see what it does and does not tell you.
The cap and diff-only mode buy convergence by blinding the reviewer. A diff-only re-review cannot see that a new change breaks an invariant defined elsewhere; a three-round cap can end the loop while a real bug remains. The post says the bug class is flagged and old code goes to nightly runs, which is the right mitigation, but it reports no measure of bugs missed after the changes. “Fewer rounds” is a cost metric; the quality metric (escaped defects per PR before and after) is absent. Ask for both before copying the policy.
The principle generalizes beyond reviewers: any agent loop with a critic needs a stop rule, a scope rule and a place for out-of-scope findings. Without those, the critic’s recall becomes the loop’s runtime.
An eval built from incidents measures recall on bugs that escaped and were later confirmed. That is the right seed, but it is survivorship-biased: bugs nobody ever noticed are not in it, and the set drifts toward bug types the existing pipeline already misses in the specific way that made them incidents. It also invites overfitting: a prompt tuned to 100% of last quarter’s incidents may be tuned to those incidents. Hold out a slice you never tune on, and refresh it.
Author response is a noisy label. When every finding must be resolved with a reason, “wrong” and “intentional” are partly social choices, and a hurried author may mark “fixed” to clear a gate. Spot-check a sample with an expert, as the incident set does. Compare to the Evals: error analysis first course in this library: read the failures before trusting the aggregate.
Do not skip the false-alarm run. Known-clean PRs are the only place you learn the cost side of a prompt change.
If you land a small or medium change and it breaks, you usually fix it. The norm works because changes are small, everything is reversible (flags, reverts, staged releases), blast radius sets the review bar, regressions surface fast through telemetry and internal use, and incident reviews examine the system rather than blame people. Remove any of those preconditions and the norm turns into scapegoating.
Engineers ask Claude to write an explainer for a slice of code (for example how managed settings are retrieved) and sometimes a small visual artifact, arguing this teaches more than line-by-line reading of trivial diffs. I find this plausible for learning a module you are about to change and unproven for maintaining awareness of what others changed, which was the human-review function Bacchelli & Bird observed. Measure it: do on-call engineers now understand recent changes better or worse?
Agents make starting work cheap, so some PRs should not land. Raise direction concerns early in the thread, not as blocking comments on a finished PR, and treat closing a quickly generated PR as normal, not a loss.
There are no circuit breakers yet that tighten approval requirements when revert or incident numbers move, and engineers want them before more code moves to zero-human merges. That is the most important sentence in the post. Without it the guardrail metrics are only a dashboard. I would build the breaker first: a rule such as “if reverts in a path exceed X in 7 days, that path drops one tier of autonomy until reviewed.”
CLAUDE.md, formatter and linter hooks).REVIEW.md; add a security pass; require threads resolved.# REVIEW.md (payments/)
Failure here means: wrong amounts, double charges, silent retries.
Check: every money path has an idempotency key; rounding is explicit.
Skip: generated/, *.snap, docs/ (owned elsewhere).
Nits: at most 3 per review; style belongs in the linter.
Every finding: file:line and the failing scenario in one sentence.
Start the summary with: "N important, M nits, K pre-existing".REVIEW.md to your riskiest directory, read from the base branch.The claude.dev post (published 15 Oct 2026) was read in full on 16 Oct 2026. Supporting facts come from the Claude Code Code Review docs and Anthropic’s “Redesigning source control” post; the Bacchelli & Bird finding is from my background knowledge of the ICSE 2013 paper and is not re-fetched here. Arithmetic (12×, 3.4×, 2,500 hours, 760 runs) is mine. Simulator cases are illustrative, and the tier assignments marked as my position are policy choices, not Anthropic’s.