The Agent-Era Engineering Org: Team Shape, Ladders and Reviews When Agents Write Most of the Code

From tickets to delegated runs

For thirty years the unit of engineering work was the ticket. One engineer picked it up, wrote the change, opened a pull request, someone else reviewed it, and it merged. Generation was the slow step. Everything downstream of it, from review to release management, was sized on the assumption that a person could only produce so much per day.

That unit is gone in any team that has adopted coding agents properly. The new unit is the delegated run: a prompt, a context bundle and a harness go in; an agent works for minutes to hours; a pull request, a report or a question comes back. The engineer’s job inside that unit is to specify, supervise and verify. Anthropic’s own scale for this, borrowed from Epoch AI, calls the relevant level “AI leads”: the model “can complete most of the task end-to-end from a high-level prompt, while the human supervises.”

This was already the dominant mode in 2025. Anthropic’s Economic Index analysis of Claude Code (28 April 2025) classified 79% of Claude Code conversations as automation rather than augmentation, against 49% for Claude.ai, and 43.8% as “directive”, meaning the task was completed with minimal user interaction. The terminal product was a delegation product from the start. Eighteen months later the coordinator products in module 2 made delegation the default shape of the whole workflow.

How far it has gone: three numbers

Read the caveats before you repeat the headline. The 26% is Anthropic’s own R&D, measured by Anthropic’s own model, on a frozen task basket; the judge agreed exactly with human raters 59% of the time, which Anthropic notes is higher than the 35% human-to-human agreement. It is a prototype, and Anthropic says so. It is not the share of work agents lead in a typical company. What it does establish is the slope: from roughly zero to a quarter of the work in six months, inside the organisation best placed to make this work.

Do the arithmetic xAI did not. One hundred PRs a day across five people is 20 PRs per person per day. An eight-hour day gives 24 minutes per PR if every human looks at every one, with no meetings and no writing of their own. Nobody reads 20 pull requests a day properly. So either most of those PRs were not human-reviewed, or what was “reviewed” was something other than the diff: a test result, a coordinator’s summary, a bot’s verdict. That single number contains the whole organisational problem this course is about.

The third number is Google’s. Sundar Pichai said in April 2026 that 75% of new code at Google is AI-generated, up from 25% in 2024 and 50% in the autumn of 2025, and that engineers review and approve all of it (Semafor, 24 April 2026). Generation share tripled in two years. Review, on Google’s account, is holding. Whether it is holding because Google has the tooling or because “review” has quietly changed meaning is the question every CTO should ask of their own org, not of Google’s.

The review bill

When generation was the slow step, review capacity scaled with it for free: one engineer produced about what one engineer could review. Google met the limit first, with mechanical refactoring tools that could produce thousands of changes at once. Its large-scale-change process exists because, as the Chromium OS guide to large-scale changes puts it, “review cost often being a significant factor” in the total cost of a change, the team must prioritise the changes with the highest value. Agents put every team in Google’s 2010 position: generation is cheap, and review is the bill.

The clearest public statement of the problem is the email a developer at a twelve-year-old codebase sent to Boris Cherny, who created Claude Code at Anthropic. Cherny published the exchange on X on 10 and 11 September 2026 (the local course What to Do About Slop reads it line by line). The developer’s thesis, in his words: “It costs more time to determine if their approach is wrong than it did for them generate it.”

Two things in that reply are organisational design, not product advice. First, the split is by disposability and blast radius, both properties of the change, neither a property of who or what wrote it. That is the basis for the review tiers in module 8. Second, the head of the product says his product’s output needs a higher bar than a person’s. If the vendor says that, your policy should not say less.

The slop course offers a working definition worth adopting as policy language: slop is output produced faster than the receiving system can verify it. It is a rate problem, not a quality problem. The same change is slop or not depending on the capacity that exists to check it. That is why “the code is fine” and “this is slop” are both true in the same standup, and why the fix is organisational rather than a better model.

Four products, one shape

Between 10 and 29 September 2026 four companies shipped the same abstraction: a long-lived coordinator that does not write code, a pool of workers that do, and a surface where a human approves what comes back. The details differ in exactly the places an org designer should care about.

Add a fifth, less finished pattern: Claude Code’s agent teams, documented as experimental and off by default. One session leads, teammates run in their own context windows and coordinate through a shared task list with dependencies. The docs carry two warnings that matter for org design: agent teams “add significant token cost”, and they “don’t isolate teammates in worktrees, so each teammate should own a separate set of files.” That is a topology whose isolation is a convention, not a mechanism.

Vendor numbers, labelled as such: Cursor says new users of Projects merge 30% more PRs and that users who primarily use Projects merge six times as many. No baseline, cohort size or window is published. Treat it as evidence that the product changes behaviour, not as a forecast for your team.

Who owns the coordinator

A coordinator that can read a Slack channel, act on a schedule, spawn hundreds of workers and open pull requests is not a tool. It is a service with authority. The product launches do not tell you who in your organisation owns it, and the default, nobody, is the worst answer. Three ownership models are workable:

Blast radius at the topology level

Cherny’s rule in module 1 is per change. A topology multiplies it. Four isolation primitives appear in the table above: a branch plus a copy of the repo per thread (Claude Code Projects), a dedicated cloud machine per project (Cursor), an approved environment with fixed permissions (Codex), and no mechanical isolation at all (agent teams). The question for each is the same three-part one: what can the coordinator and its workers read, what can they write, and what can they trigger. Credentials should be scoped to the coordinator as a service principal, never borrowed from the engineer who set it up.

Three lessons travel directly to a product org. Egress is enforced by the network, not requested of the model. The automatic stop is the control that matters; a monitor that flags in 15 minutes and a human who takes 2.5 hours to pull the plug is a 2.5-hour incident. And the sandbox your coordinator runs in is your blast radius; a worker with a production database credential and a Slack trigger is one bad plan away from an outage.

Anthropic’s disclosure in the same index shows what review of agent actions looks like at fleet scale: 100% of agent actions pass through an online monitor before execution, 0.002% of more than a billion August decisions (about one in 47,000) were blocked, roughly 100,000 transcripts are flagged each week and about 50 of the highest-priority flags are escalated to human review. The review of agent behaviour is itself automated, with humans on the exceptions. That is the shape your own review function will take; the choice is whether you design it or discover it.

When a human is the coordinator

The products all keep a human-as-coordinator surface: Claude Code Projects’ overview pane shows which threads are ready for review; Codex has the /agents view; Cursor’s merge approvals show the commit subject and message the agent chose. Use that mode by default for ambiguous scope, cross-team changes, anything touching money, authentication or data deletion, and the first 90 days of adoption (module 8).

The hand-off test for letting the bot coordinate without you: the task has a machine-checkable definition of done, a rollback you have exercised, and a blast radius you would accept unreviewed. If any of the three is missing, a human stays in the coordinator seat. That is not caution for its own sake. It is the only way to keep the review bill from module 1 from arriving all at once.

The model

Three inputs. A is the number of agents an engineer supervises. P is the number of mergeable changes each agent produces per day. R is the number of changes that engineer can review properly per day. Verified throughput per engineer is the smaller of production and review: min(A×P, R). When A×P exceeds R, the difference is queue growth per day, and a queue in front of a review gate turns the gate into a formality.

R is the input people refuse to estimate, so anchor it. SmartBear’s study of a Cisco team is still the most-cited measurement of human review: review no more than 200 to 400 lines at a time, over 60 to 90 minutes, which “should yield 70–90% defect discovery,” and defect density drops significantly at inspection rates faster than 500 lines per hour. Take three hours of review a day, which is two sessions and more than most seniors give, and 20 minutes per 150-line agent change. That gives R = 9. Cross-check: 400 lines an hour for three hours is 1,200 lines, which at 150 lines a change is 8. The two anchors agree within one change.

Inputs: R = 9 (three review hours a day at 20 minutes per change). Every cell is min(A×P, 9).

Now a team. Ten engineers, each producing two changes a day before agents: 20 changes a day against a review capacity of 90, which is why review never felt like the constraint. Give each engineer four agents at three changes a day: 120 changes a day against the same 90. The shortfall is 30 a day, 150 a week, and it compounds, because the queue also eats the reviewers’ production time. The team that bought the most agents is the one whose verified throughput falls first.

Three levers move the arithmetic. Tier by blast radius: if 60% of those 120 changes are low-blast-radius and go through bot review only, the human share is 48 against a capacity of 90, with headroom of 42. Pre-review by machine: if a review bot’s triage halves the human minutes per change to 10, R becomes 18 and the break-even at P = 3 moves from 3 agents to 6. Shrink the change: SmartBear’s curve is about lines, so smaller changes raise R directly. All three are organisational choices, not model upgrades.

What the measurements say

No published study has measured teams running coordinator fleets at scale; the products are a month old. What exists measures the first wave, assistants and early agents, and it points one way: production up, verification strained.

Three levers that raise R

1. Shrink the change. DORA lists “small batches” among its seven AI capabilities precisely because large AI-generated changes are hard to review and test. Make the coordinator split work; a 150-line change gets 20 minutes of real attention, a 1,500-line one gets a skim.
2. Put a machine in front of the human. Security and bug bots (module 8 names them) turn the reviewer’s input from a diff into a triage. The SmartBear finding that author annotation sharply reduces defects applies to agents too: make the agent write the review packet, with what changed, why, how it was verified and what is risky.
3. Tier by blast radius. Humans read what can hurt. Everything else gets bot review, tests and a monitored rollout. Module 8 gives the tiers.

What the data says

Four sources, each with a different baseline. Do not blend them into one number; the baselines are the finding.

Hold the mechanism in view. The Stanford paper finds the adjustment is happening through hiring, not layoffs, which means the pipeline is being switched off at the entrance rather than drained. SignalFire’s explanation is the one most CTOs would give if asked: juniors used to do the work agents now do, and a junior’s code adds to the review queue from module 3. Both are true. Neither makes the outcome inevitable.

Put those two facts next to each other. The industry stopped hiring the people it used to train, and the leading model vendor is now spending nine figures to train deployment engineers because it cannot find them. The skill in shortage is not writing code. It is making agents work inside a real codebase, which is precisely the skill an apprenticeship built on supervision would produce.

An apprenticeship built on supervision

The structure is deliberately old-fashioned. It is how trades have always transferred judgment: supervised practice with the consequences held by someone senior, then widening scope. The thing being apprenticed has changed from writing to verifying. The method has not.

Ladders that reward harness design and verification

Most ladders written before 2024 reward output and scope: ships features, owns a service, designs systems, sets direction. In an agent org the first of those is now mostly the agent’s. Rewrite the criteria so each level is defined by what the person is accountable for verifying and what they build for others to verify with.

What changed, and when

Notice the convergence. Meta and Google both replaced an algorithm round with a round on an existing codebase, with the assistant present, scored on how the candidate directs and checks it. That is a decomposition-and-verification interview wearing a coding-interview costume. Anthropic’s reversal within two months is the more instructive story for a smaller company: the first instinct, to ban the tool and see the “real” person, did not survive contact with how people now work.

The four signals that predict performance in a supervised-agent org

If the unit of work is the verified change with a named owner, the interview should predict whether the candidate can own one. Four signals do that; keystroke speed and algorithm recall do not.

1. Decomposition. Give a vague product ask and thirty minutes. Can they turn it into three to five runs, each with a crisp, machine-checkable definition of done and an explicit list of what is out of scope? Watch for the candidate who writes the acceptance test before the prompt.
2. Verification design. Show a finished change and ask what they would check before merging, what they would automate, and what they would refuse to merge without a human reading it. Score the ordering, not the list.
3. Critical reading of agent output. Hand over an agent-written PR with two planted problems: a test that asserts the behaviour of its own mocks, and a quiet relaxation of the spec that makes the task easier, which is the exact failure the developer in the slop exchange described. Candidates who find both in twenty minutes are rarer than you think and are worth more than the ones who can write the change from scratch.
4. Harness literacy. Give them a CLAUDE.md or AGENTS.md from a real repo with two instructions that were right for a 2025 model and are wrong now. Can they say which and why? This is the skill module 4’s L4 is built on.

Score each round on a three-point anchor written before the loop runs, and debrief in the order the rounds ran, with the agent-PR review first because it carries the most signal. Require every interviewer to read the candidate’s run log before the debrief; interviewers who have read it score the decomposition round far more consistently than those who have not. Keep the loop to four or five rounds. The signals above compound, and a sixth round adds fatigue, not information.

Take-homes with agents allowed

A take-home that bans agents in 2026 tests a skill the candidate will not use on the job and is unenforceable anyway. A take-home that allows them with no limits tests the candidate’s credit card. The workable form has four rules:

• Agents are allowed and expected. Say so in the brief.
• Submit the run log with the PR. Transcripts or session exports, not screenshots. You are grading how they directed and checked, not what came out.
• Set a budget. A token cap or a time cap, for example one hour of agent time, and say that exceeding it is a negative signal.
• Score the verification note. Require a short statement of what was verified, how, and what was not. The strongest candidates write the longest “not verified” sections.

What to stop asking

Drop whiteboard algorithm recall, “implement an LRU cache”, language trivia, and anything whose answer a model produces in one shot. Drop “how many PRs did you ship last quarter”; module 6 explains why that number means nothing. Keep system design, but add the agent harness and the coordinator’s permissions as components the candidate must design. Keep live debugging, with the tool present. Keep the behavioural round, and change its questions to be about verification that failed rather than about delivery that succeeded.

Why the output proxies broke

PR count, lines changed and commit frequency were always weak measures. They were tolerable because producing them cost human effort, so a person who produced a lot had at least done a lot. An agent fleet produces them for tokens. Worse, the field data in module 3 shows what maximising them looks like at population scale: 31% more PRs merging with no review, incidents per PR more than tripled, code churn up 861% (Faros, March 2026), duplication rising while refactoring falls (GitClear), throughput up with stability down (DORA 2025). Any review process that rewards the proxies is paying for those outcomes.

SPACE was written for a world where Activity was a weak but real signal. In an agent organisation the Activity dimension is not weak; it is measuring the agent. The other four still work, and Performance-as-outcomes is now the whole game.

What to measure instead

The last item is View 1 from the slop exchange turned into a question. The developer who wrote to Cherny did not ask for every change to be reviewed; he asked that “the person submitting the code should be able to explain it.” Ask that directly, anonymously, every quarter, and plot it against unreviewed-merge share. When the two lines cross, you have found the team that has quietly become View 2.

The quarterly planted-defect exercise

Review quality needs a control, because comment counts and latency can both be gamed by a reviewer who writes “looks good, nice work” quickly. Once a quarter, the coordinator owner takes three real agent-written changes from the previous month, re-opens them on a branch with one defect planted in each (a test that asserts its own mocks, a boundary condition removed, a permission check loosened) and routes them through the normal review queue without flagging them. Record who caught what. Publish the team’s catch rate, not individuals’, and use the individual results only in the owner’s conversation with each reviewer. Three changes a quarter cost a senior about two hours to prepare and give you the only number about review quality that a fleet of agents cannot move. If the catch rate stays below two in three for two quarters, the tiering in module 8 is wrong for that team; that is a finding about the gates, not about the people.

Calibration pitfalls

The generator/verifier split. In calibration the engineer who ran twelve agents and merged forty changes looks productive. The engineer who caught three bad merges among them looks slow. If your packet counts changes, you will promote the generator and lose the verifier, and module 3 says the verifier was the constraint. Fix it structurally: a verified change has an owner and a verifier of record, both named in the packet, and catches are listed as contributions with the defect they prevented.

Token spend as a halo. Spending more is visible in the billing export and feels like effort. It is not output. Present spend only as the denominator of cost per verified change.

Agent-caused incidents. Run them blameless, as always. But the accountability question is not “why did the agent do that”; it is “what verification should have caught it, and who owned that tier”. Attribute the verification gap, not the agent’s error, and change the tier or the gate.

The review packet, one page

The packet is what calibration reads, so its shape decides what calibration rewards. Keep it to one page per person with six blocks, in this order: defect escape per verified change (owned, trailing 90 days, by tier) with the team median beside it; verified changes owned and verified-of-record, as two separate counts; outcomes moved, with the metric and its before and after; catches, each with the defect it prevented; harness contributions, each with its measured effect; and the perceptual items. Token spend appears once, as the denominator of cost per verified change, never as a standalone line. A manager who cannot fill the first block for someone has not been running the measurement, and that is the manager’s finding, not the engineer’s.

The price list, 13 October 2026

All prices are Anthropic’s list prices per million tokens on the Claude API, from the platform pricing page, checked on 13 October 2026. The three 5.5 models shipped on 22 September (Opus), 28 September (Sonnet) and 7 October 2026 (Haiku).

Two features of the table matter more than the headline prices. Cache reads on the 5.5 models cost 5% of input, so a long-running agent whose context is mostly cached pays far less than the list price suggests. And Haiku 5.5’s long-prompt tier means an agent that lets its context grow past 100,000 tokens pays five times more per request from that point; a cheap model with a sloppy harness is not cheap. The spread from Opus to small-prompt Haiku is 40× on both input and output.

The cost of a run

Stated inputs: a delegated run consumes 2 million input tokens, 80% of them cache hits, and produces 100,000 output tokens. These are round numbers for a substantial multi-step run; your own billing export will give you better ones within a week.

The cost of a verified change

Tokens are the small part. Stated inputs for the human side: Levels.fyi’s 2025 US medians for total compensation by level, a 1.3× fully-loaded multiplier (benefits, payroll tax, equipment, space; my assumption, adjust to your finance team’s number) and 220 working days a year.

Now one verified change. Assume 1.5 Opus runs per merged change (one run plus half a retry on average) and 20 minutes of a senior’s review. Tokens: 1.5 × $3.92 = $5.88. Review: 20 minutes at $230 an hour = $76.82. Total $82.70, of which tokens are 7.1%. Make the review an hour and the total is $236.33 with tokens at 2.5%. Replace Opus with Sonnet and the token share at 20 minutes falls to 3.7%. In every case, the human verification minutes are the cost of a verified change, and the token line is a rounding error on it.

Presenting it to a CFO or a board

Finance will accept tokens as a line item. What it will not accept, and should not, is tokens presented as a substitute for people with no account of what the people were for. The slide that works has five lines and one trend:

The trend to show is cost per verified change against defect escape, quarter by quarter. If the first falls while the second holds, the agents are paying for themselves. If the first falls while the second rises, you are buying incidents at a discount, and module 3 predicts that is the default outcome of adding agents without raising R.

When buying agents beats hiring, and when it does not

The shape of the 90 days

Ninety days is about 13 weeks. Do not reorganise in week one. The first 90 days change policy and measurement, not boxes on a chart; the boxes follow once you know your R and your tiers. The plan is two weeks of diagnosis, six weeks of a single-team pilot, and roughly five weeks to decide and scale.

Review gates by tier

This is Cherny’s split turned into policy. Tier assignment is the change owner’s call and is audited; an owner who tiers down to avoid review loses the privilege.

The bots you can turn on this quarter

Note what the bots have in common: none of them merges. They convert a diff into a triage for a human, which is the second lever from module 3. The one that changes the organisation most is the rollout monitor, because it moves part of verification from before merge to after deploy, where a tier 1 change can be allowed to prove itself in production under a monitor instead of in a queue.

The policy document, in one page

1. Unit of work: the verified change with a named owner (module 1).
2. Tiers and gates: the table above, with the audit rule.
3. Coordinators: ownership model, the read/write/trigger answers for each, credentials scoped to the coordinator, an audit log that answers “what did it do last night” (module 2).
4. Capacity: each team publishes its R and its break-even agent count and revisits them quarterly (module 3).
5. Budget: a token budget per engineer and per coordinator, reported as cost per verified change, never as a standalone total (module 7).
6. Measurement: the six measures from module 6, with defect escape per verified change at the top of every packet.
7. People: the apprenticeship and ladder from module 4; the loop from module 5.
8. Containment: egress enforced by the network; an automatic stop for every coordinator; a drill once a quarter.

What to say to whom

• To the team: no headcount decision rides on the pilot. The pilot decides policy. Say it in writing, because the Shopify and Duolingo memos of 2025 are what everyone will assume you mean.
• To the CFO: the five-line slide from module 7, with cost per verified change and defect escape as the trend. Tokens are a multiplier on verifiers, not a substitute for them.
• To the board: one chart, cost per verified change against defect escape, and one sentence about containment: who can stop a coordinator and how fast.

Failure modes and their signals