Harnesses That Fit the Task: Dynamic Workflows and Measured Agent Loops

Claude Code can now write its own multi-agent harness; Anthropic used a Slack-driven agent loop to speed claude.ai up 3× in two weeks. What each does, what it costs, and what neither post proves.

8 modules
2 commit-first simulators
~75 min
Oct 2026 sources

Spine texts (claude.dev): Dynamic workflows in Claude Code · How we made claude.ai 3x faster

How to read this course

Both posts are written by Anthropic about Anthropic’s own tools, and both are results-first: the workflows post gives patterns and two token counts but no comparison against a static baseline; the claude.ai post reports a dazzling aggregate but no cost, no revert count and no counterfactual. I separate what the source says, what it leaves unmeasured or self-serving, and independent evidence (Anthropic’s own multi-agent research system write-up and Building effective agents).

Modules 1, 3, 4 and 6 make you commit to an answer before the reveal. Related courses in this library: Loop Engineering (the Cherny view of the same pattern) and The Harness Is the Policy.

Course Modules

  1. Why one context failsFoundations + predict
  2. Six harness patternsPatterns
  3. The Pattern ChooserSimulator
  4. What the evidence supportsCritique + predict
  5. The measured loop: claude.ai in two weeksCase study
  6. The Measurement DeskSimulator
  7. Designing yours: decision table & failure modesMethod
  8. Quiz, flashcards & Monday checklistReview
1

Why one context fails

The case for harnesses — and its price tag
Learning objectives
  • Name the three single-context failure modes the post uses to justify subagents
  • Estimate what a fan-out workflow costs in tokens from its width
  • State when not to use a workflow

The pitch

Dynamic workflows let Claude Code write and run a JavaScript file that orchestrates subagents with three primitives: agent() to launch one, parallel() to fan out, and pipeline() to chain stages. A workflow can pick the model per agent (Opus, Sonnet, Haiku) and the isolation level (a git worktree, or remote). “Dynamic” means the harness is written for this task rather than being a generic loop: where a static harness runs the same five web searches every time, a dynamic one might inspect the billing code, verify features against provider docs, price at real volumes and add an adversarial check before recommending.

Three failure modes of a single context

  • Agentic laziness — the model declares done early.
  • Self-preferential bias — it grades its own output generously.
  • Goal drift — repeated summarization erodes fidelity to the original objective.

Separate subagents with fresh, isolated contexts address each: a verifier has no authorship to defend; a worker sees only its slice of the goal; a loop can keep launching agents until a completion condition is met instead of a fixed count.

Published exampleAgentsTokensTime
review-changes14482K6 min 12 s
deep-research221.1M11 min
Predict

Commit a number first

From the table: the deep-research workflow used 22 agents and 1.1M tokens. What is the average number of thousand tokens per agent? Commit a number, then see what it implies for how you budget a workflow.

Enter a number to unlock the reveal.
Where I push back

The three failure modes are real but the cure is not free. A fresh-context verifier from the same model family shares the generator’s blind spots; independence of context is not independence of error. Adversarial verification reduces self-flattery, not correlated mistakes. The post gives no false-positive or miss rate for any verifier pattern. Where a deterministic check exists (tests, type-check, a diff against a reference), prefer it over a verifier agent at every stage it can reach.

The post is honest about cost (“significant token costs”, “most standard development work doesn’t benefit from five-reviewer panels”). Take that seriously: the default for ordinary coding should stay a single agent.

Takeaways
  • Single-context failures: laziness, self-grading bias, goal drift.
  • Workflow cost ≈ width × per-agent context (34–50K in the published examples).
  • A verifier agent adds independence of context, not of error.
2

Six harness patterns

Classify, fan out, verify, generate, tournament, loop
Learning objectives
  • Describe the six patterns and the failure each addresses
  • Map real tasks to patterns
  • Recognize the quarantine pattern for untrusted input
PatternShapeUse whenMain risk
Classify-and-actroute by type to a specialistheterogeneous inputsmisroute; classifier becomes a bottleneck
Fan-out & synthesizeN parallel agents → barrier → merge structured resultsindependent subtasks (callsites, sources, modules)unstructured outputs make the merge lossy
Adversarial verificationseparate agent checks output against a rubricclaims, diffs, reportscorrelated errors; rubric quality
Generate & filtermany ideas → score → dedupenaming, design optionsthe filter encodes your taste; garbage rubric = garbage pick
Tournamentpairwise judges eliminate competitorsranking 1,000+ items; competing approachesjudge noise; O(n) comparisons of cost
Loop until donekeep spawning until a completion condition holdsopen-ended workno true stop condition → runaway spend

These map almost one-to-one onto the five workflows in Building effective agents (routing ≈ classify-and-act, parallelization ≈ fan-out, orchestrator-workers ≈ fan-out with a planner, evaluator-optimizer ≈ verify/loop). What is new in the dynamic version is not the patterns — it is that the model writes the orchestration code for the specific task.

The applications the post lists

  • Migrations/refactors: decompose by callsite/test/module; parallel fixes; adversarial review before merge.
  • Deep research and deep verification: fan out searches; one agent per claim to check; optional source auditors.
  • Sorting: tournament comparisons instead of cramming 1,000 items into one context.
  • Rule adherence: one verifier per CLAUDE.md rule, a skeptic to cut false positives.
  • Root cause: independent hypotheses from disjoint evidence (logs, files, data) tested by panels.
  • Triage at scale (quarantine): unprivileged reader agents classify and dedupe untrusted content; separate actor agents with privileges act only on the readers’ summaries.
  • Taste decisions, evals, model routing (a classifier agent picks Sonnet vs Opus).
The quarantine pattern, stated precisely

It is a prompt-injection control: any agent that reads untrusted text gets no tools with side effects; any agent with side effects reads only the sanitized, structured summary. It works only if the summary schema cannot carry instructions (enumerations and IDs, not free text). That is the single most security-relevant idea in the post, and it gets two sentences.

Takeaways
  • The six patterns are old; the novelty is model-authored orchestration.
  • Quarantine = untrusted readers without privileges, privileged actors without untrusted text.
  • Structured, schema-bound hand-offs are what make fan-out merges and quarantine safe.
3

The Pattern Chooser

Five tasks. Pick a harness, commit, then see the reasoning.
Simulator

Pattern Chooser

Takeaways
  • Parallel + checkable → fan-out with gates; ranking at scale → tournament.
  • Untrusted text and side-effect tools never share an agent.
  • The simplest harness that meets the check wins.
4

What the evidence supports

Independent numbers, and the missing control
Learning objectives
  • State what Anthropic’s own multi-agent research system measured
  • Explain why token spend confounds multi-agent claims
  • Say what experiment would actually test a workflow

The independent evidence

Anthropic’s multi-agent research system (orchestrator Opus, Sonnet subagents) reported a 90.2% improvement over a single-agent Opus on an internal research eval. Its cost: agents use ~4× the tokens of chat, multi-agent systems ~15×. And the finding most people skip: token usage alone explained about 80% of the performance variance. They list poor fits — most coding tasks and anything needing shared context — and early failure modes: 50+ subagents for a trivial query, duplicated work from vague task descriptions, picking SEO spam over primary sources.

Predict

Commit a number first

If agents use ~4× chat tokens and multi-agent systems ~15× chat tokens, how many times more tokens does the multi-agent system use than a single agent?

Enter a number to unlock the reveal.
The experiment neither post runs

Compare a workflow against a single agent given the same token budget (e.g. best-of-N, or one agent with a 3.75× longer budget and the same deterministic checks). Only if the workflow wins at equal spend has structure, rather than spend, earned its place. The dynamic-workflows post reports two runs and no comparison at all; each pattern is justified by narrative.

Tension to resolve: the workflows post sells code migrations and refactors; the research write-up says most coding is a poor fit. The reconciliation is that migrations are parallel with machine checks, whereas open-ended feature work needs shared context. Treat “coding” as two different workloads.

Static vs dynamic is also unmeasured. Letting the model write the harness adds a failure mode: a badly written orchestration (wrong barrier, lossy merge, no stop condition) that you did not review. Save and review workflows that matter (the post’s “s” key and ~/.claude/workflows); treat a generated workflow like generated code.

Takeaways
  • Published multi-agent gains came with ~3.75× tokens vs one agent; token spend explained ~80% of variance.
  • Require an equal-budget single-agent baseline before crediting structure.
  • Review and version workflows you rely on; model-written orchestration is code.
5

The measured loop: claude.ai in two weeks

Measurement, ratchet, flags — and a pile of unreported denominators
Learning objectives
  • Describe the loop each Slack thread ran and why each step exists
  • Explain dual validation of a metric and the ratchet
  • Separate what Claude did from what the humans steered

The result

In two weeks (August 2026) the claude.ai and desktop apps got roughly 3× faster across thirteen user journeys. Examples: fresh web load 3.1 s → 0.55 s (5.6×); desktop cold start 6.3 s → 3.3 s; loading an existing conversation 2.4× on web and 2.8× on desktop; Cowork message send 928 ms → 48 ms (19×). By day three they had hit 12 of 13 original targets and set new ones; at peak 150+ threads ran at once and 200+ changes landed on the busiest days.

The insight: measuring makes it tractable

Once a metric exists, Claude can hill-climb it. The team built several complementary benchmarks: Valgrind-based instruction counting for JavaScript hot paths (node --predictable), V8 precise-coverage call counts, React commit counts, style-recalculation counts and DOM-mutation counts. Crucially, each metric had to be validated twice: that it tracked wall-clock time, and that optimizing it produced real gains. Their example: cutting instructions 48% on message-tree assembly improved wall-clock 78%, which earned the metric trust.

The loop, one Slack thread at a time

  1. A human flags a slow flow, often with a screen recording.
  2. Claude traces it and writes a reproducing benchmark.
  3. If lab results look promising, Claude opens several reviewable PRs.
  4. Changes ship behind feature flags.
  5. Claude watches field data after rollout.
  6. Ratchet: keep wins by lowering the benchmark threshold so regressions fail CI; revert the rest.

Guardrails

  • Unit tests before the change; user-visible work behind flags; automated code review plus at least one human approval.
  • Pixel-exact layout tests across 14 viewport sizes for the static composer; a keystroke test that typed through the static-to-React handoff and failed on any lost or reordered key; layout-shift telemetry to a tenth of a pixel.
  • Staged rollout: employees → 1% → everyone (this caught a Chrome speculative-loading resize quirk).
  • ~200 feature flags created, each classified by Claude as kill-switch or gradual ramp; over half already cleaned up.

What the humans did

Three things: push ambition (“we have the power to do anything. please be braver” — Claude by default hedges and pads estimates), aesthetic judgment (each thread had a named human owner reviewing before/after screenshots: skeleton at 0 ms or after 500 ms? is a word-fade worth a fifth of the frame budget?), and scope control (narrow threads; rejected a 2 ms/message win that needed a complex build plugin).

Arithmetic I re-checked

3.1 ÷ 0.55 = 5.6× and (3.1−0.55)/3.1 = 82% reduction — both match. 928 ÷ 48 = 19.3×. The headline “3×” is therefore an aggregate over 13 journeys whose individual gains range from 1.5× to 19×; it is not a typical journey and the post doesn’t say how it aggregates.

Where I push back

What is not in the post: total token/compute spend for 150 concurrent threads; the number of reverted changes (“unsuccessful ones were reverted” — how many?); the human review hours; and any baseline for what the same team would have achieved in two weeks with the same measurement discipline and no agents. “Tens of thousands of user-hours saved” is an unshown calculation. Selection bias is built in: a post celebrating a sprint does not report the failed sprints.

Where it probably generalizes: performance is close to an ideal agent task — a scalar metric, cheap reproduction, mechanical rollback. The load-bearing idea is validated metric + ratchet + flags + staged rollout, not “150 agents.” Without a scalar you trust, the same loop optimizes the wrong thing faster (the ratchet locks in a mistake).

Lab-to-field gap: instruction counts and headless frame stepping are proxies; the post’s defense (field telemetry after each flag ramp) is the right one, and it is the part to copy first.

▶ Animated recap · 1:40
Two hosts walk the measured loop — a validated metric, five steps and the ratchet — then the before/after numbers, all from the module’s own figures.
Takeaways
  • Make the thing measurable, validate the metric against reality, then ratchet.
  • Flags, staged rollout and field telemetry are what made 200 changes a day survivable.
  • Humans supplied ambition, taste and scope; the report omits cost and reverts — ask for those in your own version.
6

The Measurement Desk

Five calls an owner of an agent loop has to make
Simulator

Measurement Desk

Takeaways
  • Validate the proxy, ramp on field data, price complexity, set ambition, retire flags.
7

Designing yours: decision table & failure modes

When to use what — and the spec to write first
Learning objectives
  • Pick between single agent, workflow and measured loop for a task
  • List the failure modes of each and a mitigation
  • Write a one-paragraph spec for a workflow you would run
SituationUseGate that must exist
Small, well-scoped change; tests existsingle agentthe tests
Parallel slices, deterministic per-slice checkfan-out workflowper-slice tests; merge schema
Breadth-first research, claims to verifyfan-out + verifier per claimcitations; source audit
Subjective choice at scalegenerate-and-filter / tournamenta rubric you have calibrated on hand-labelled pairs
Untrusted input plus side effectsquarantine (reader / actor)a schema with no free-text channel
Open-ended improvement of a scalarmeasured loop (benchmarks, flags, ratchet)a validated metric; field telemetry; rollback

Failure modes to design out

  • Runaway width/cost — early research agents spawned 50+ subagents for trivial queries; embed scaling rules and token budgets.
  • Vague delegation — duplicated work; each subagent needs an objective, output schema, and boundaries.
  • Correlated verifiers — prefer deterministic checks; use heterogenous prompts or a different model for the skeptic.
  • Metric gaming — a benchmark can be optimized in ways users don’t feel; keep a human-reviewed before/after and a field metric.
  • Merge loss — synthesis from free-text results; use structured outputs.
  • Orphaned autonomy — many threads, no owner; the claude.ai sprint named a human owner per thread.

A workflow spec you can copy

Goal: <one sentence + completion condition>
Pattern: fan-out by <unit>, then adversarial review, then merge
Worker: model=Sonnet, isolation=worktree, output=JSON{unit,status,diff_summary,evidence}
Gate: per-unit tests must pass; reviewer (fresh context, rubric below) must fail to refute
Rubric: <3-5 checkable questions>
Budget: max <N> agents, <T> tokens; stop and report if exceeded
Baseline: run the same task single-agent with the same token budget; compare pass rate
Takeaways
  • Match harness to task shape and name the gate before launching.
  • Cap width and tokens; require structured outputs.
  • Run the equal-budget baseline at least once per workflow type.
8

Quiz, flashcards & Monday checklist

Commit an answer, then see the explanation

Quiz

Flashcards (click to flip)

Monday checklist

  1. Before a workflow: write the gate (tests, schema, rubric) and the token/agent cap.
  2. Use structured outputs for every fan-out hand-off.
  3. Run each new workflow type once against a single agent with the same token budget.
  4. Split untrusted readers from side-effect actors in any triage workflow.
  5. Pick one scalar you can trust; validate it against real time before any loop optimizes it.
  6. Track cost, reverts and human-review time for every agent loop you run — the numbers the post omitted.
Sourcing note

Both claude.dev posts were read in full on 1 Oct 2026. Supplementary numbers (90.2%, 4×/15×, 80% variance, failure modes) are from Anthropic’s multi-agent research system write-up, and the five workflow definitions from Building effective agents. Arithmetic is mine. Simulator cases are illustrative.