Ashita Orbis

The Pruning Principle: When Scaffolding Becomes Dead Weight

This article examines the architectural claim that scaffolding built around foundation models can become a performance bottleneck as those models improve, and tests that claim against current evidence from agent design guidance, the SWE-bench leaderboard, and published Claude Code and Codex CLI design notes. The thesis survives in a weaker, more disciplined form: cognitive scaffolding should be aggressively re-audited as models gain capability, but environmental, safety, and observability scaffolding rarely belongs on the chopping block. Treating "prune scaffolding" as a universal ratchet is an error the evidence does not support.

Coverage note: verified through May 6, 2026.

Why the framing matters

A recurring argument in current agent-engineering writing runs like this: foundation models have improved so quickly that much of the orchestration we built around them in 2023 and 2024 — the elaborate planners, the reflective sub-loops, the "think step by step" preambles, the safety preprompts, the structured-output coercion — was solving problems the current generation no longer has. If those layers persist, they are not merely inert; they actively degrade performance by polluting context, adding latency, distorting reasoning, and obscuring what the model would otherwise do. The prescription is to ratchet scaffolding complexity down as model capability ratchets up.

This framing has real empirical pull. Anthropic's Building Effective Agents is widely (and correctly) read as a defense of starting simple and adding complexity only when it earns its place. The mini-SWE-agent project — roughly a hundred lines of Python wrapping a model in a bash loop — sits at or near the top of the SWE-bench Verified leaderboard, beating or matching agents with substantially heavier orchestration. Anthropic's own Claude Code design retrospective admits that early scaffolding (the TodoWrite tool, persistent reminders) became counterproductive once models were strong enough to manage their own working memory.

But the framing also fails in characteristic ways. It conflates several distinct categories under one word — "scaffolding" — and then prescribes a single response. It overreads the mini-SWE-agent result, which is a comparison of language models under a deliberately stripped harness rather than a verdict on production architecture. It ignores that the same SWE-bench history contains the opposite lesson for older models: the original SWE-agent paper showed that a well-designed agent-computer interface improved GPT-4 Turbo's task success by 64% relative to a shell-only baseline. And it elides the structural difference between scaffolding that puppeteers cognition (which models are increasingly able to do without) and scaffolding that handles permissions, audit, recoverability, sandbox boundaries, and verification (which model improvement does not retire).

The corrected principle is narrower: as models improve, scaffolding that encodes yesterday's model limitations should be suspected, tested, and removed when controlled evaluation shows no current benefit. Scaffolding that handles environment, safety, observability, or organizational coordination is governed by different forces and rarely becomes "dead weight" in the same sense.

What counts as scaffolding

The first source of confusion in this debate is taxonomic. The word "scaffolding" gets used for at least five different things, and the right answer to "should we prune it?" depends on which one is in front of you.

Category Examples Primary justification Pruning sensitivity
Cognitive scaffolding Manual chain-of-thought prompts, structured-thought templates, planner wrappers that force the model to restate the task, role-play preambles, output-format coercion via prompt Compensates for limits in the model's reasoning, planning, or instruction-following High — directly affected by model improvement
Environmental scaffolding Tool definitions, agent-computer interfaces, file system affordances, search/grep/edit primitives, MCP servers, sandboxes Gives the model contact with reality; defines the action space Low — generally improves with stronger models because the model uses tools more skillfully
Verification scaffolding Test harnesses, evaluator-optimizer loops, judge models, structural validators, retry-with-critique loops Confirms outputs against ground truth; catches errors before they propagate Mixed — better models reduce some retries, but verification value is largely model-independent
Safety scaffolding Approval prompts, sandboxed execution, permission policies, blast-radius limiters, irreversible-action gates, audit logs Manages execution authority and contains damage from any agent action, including correct ones Low — model improvement may increase the need as agents act more autonomously
Organizational scaffolding AGENTS.md, skills, durable instructions, project conventions, reusable subagents, shared evaluation suites Encodes team knowledge, compliance requirements, and learned-from-failure patterns Low — these are knowledge artifacts, not model crutches

The pruning principle, stated cleanly, applies most strongly to the first column. It applies more weakly and conditionally to the third. It does not straightforwardly apply to columns two, four, and five — which are where most production complexity actually lives.

This taxonomy matters because the most cited evidence for the pruning principle (mini-SWE-agent on SWE-bench) is sometimes read as evidence that columns two through five are also dispensable. They are not. Bash itself is environmental scaffolding — a high-bandwidth, general-purpose, well-understood agent-computer interface. The mini-SWE-agent project still has a ReAct loop, a task prompt template, a Docker evaluation environment, observation formatting, step and cost limits, and a submission protocol; what it lacks is cognitive scaffolding inside the prompt and orchestration scaffolding around the loop. That is a much narrower claim than "frameworks are dead."

What Anthropic actually said

The architectural argument most commonly attached to the pruning principle traces back to Anthropic's "Building Effective Agents" (December 2024). The original text is more measured than the slogan it has spawned.

Anthropic's stated guidance is to start with the simplest workable design — often a single LLM call augmented with retrieval and in-context examples — and to add agentic complexity only when measurement shows it improves outcomes. The piece classifies common patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer, autonomous agents) and is explicit that each pattern has tasks where it earns its place. It warns that "agentic systems often trade latency and cost for better task performance" and that frameworks like LangChain or LlamaIndex can hide what the model is actually doing, making debugging harder and tempting teams toward unnecessary abstraction. But it does not argue that complexity should ratchet down with model improvement. It argues that complexity should pay rent — at every model generation, and at the time it was first introduced.

The distinction is load-bearing. "Pay rent" is a cost-benefit test applied per-component. "Ratchet down" is a directional claim about the trajectory of a system over time. The first is straightforwardly defensible. The second is a stronger claim that requires evidence the document does not supply.

The strongest pruning-adjacent claim Anthropic makes in that piece is: when you add complexity, you should be able to point to a measured improvement that justifies it; if you cannot, simpler is better. The contrapositive — as models improve, previously-justified complexity may stop justifying itself — is implied but not asserted. It is reasonable inference, not stated doctrine.

A later, sharper formulation appears in Anthropic's Claude Code design retrospective, discussed below: certain tools introduced to help early Claude Code became constraining as the underlying model improved, and the team had to redesign them. That post is where the empirical case for ossified scaffolding is strongest. But even there, the response was rarely deletion; it was migration to a different layer. The complexity moved.

The mini-SWE-agent result and what it actually shows

The clearest empirical anchor for the pruning principle is the mini-SWE-agent project. The SWE-bench Verified leaderboard uses mini-SWE-agent in a minimal bash environment as its reference setup for "apples-to-apples LM comparison" — explicitly to remove confounds from competing harnesses so that the language model's contribution can be isolated. The mini-SWE-agent repository describes the core agent class as roughly 100 lines of Python, with bash as the only tool, and reports SWE-bench Verified scores above 74%.

As of May 6, 2026, the public leaderboard shows the top mini-SWE-agent entry at 76.8% Verified, with the top all-agent entries at 79.2%. The headline is "minimal scaffold competitive with framework-heavy approaches." It is not "minimal scaffold dominates." The gap is small but consistent, and it points away from the strongest version of the pruning thesis.

Several things follow from looking at the result carefully:

The setup is intentionally minimal for benchmarking, not for production. SWE-bench's purpose in adopting mini-SWE-agent as the reference harness is to compare models, holding the agent layer constant. Reading this as architectural advice for production agents is a category error. It is closer to comparing engines on a dynamometer than to comparing cars on a road.

A small absolute gap can be a large practical gap. The gap between 76.8% and 79.2% on SWE-bench Verified is 2.4 points absolute and roughly 11% relative on the failure axis (23.2% failure versus 20.8% failure). For some workflows that is rounding error; for others — automated bug-fix systems running thousands of issues — that gap is the difference between viable and not. Using leaderboard proximity as evidence that "frameworks add nothing" understates the residual room frameworks occupy.

The benchmark itself has been called into question. OpenAI's February 2026 post Why we no longer evaluate SWE-bench Verified argues that test flaws and contamination have eroded SWE-bench Verified's value as a frontier coding signal. Even if the leaderboard ordering is informative about relative system effort, treating it as a settled measure of agent quality is shaky.

The same paradigm produced the opposite result for weaker models. The 2024 NeurIPS SWE-agent paper reports that on SWE-bench Lite with GPT-4 Turbo, the custom SWE-agent ACI scored 18.0% versus 11.0% for a shell-only baseline — a 64% relative improvement attributable entirely to interface design. The same comparison run on a substantially stronger 2026 model would likely close most of that gap, but the historical fact remains: agent-computer interface design has at certain model-capability thresholds been worth a great deal. The current convergence of bash-only and framework-heavy approaches is a story about current model capability, not a timeless truth about agent design.

The defensible empirical claim from the mini-SWE-agent result is therefore narrower than "scaffolding becomes dead weight." It is: under sufficiently capable current models, well-chosen general-purpose interfaces (like bash with a competent test environment) can match or nearly match heavier orchestration on at least one important coding benchmark, and the marginal value of additional cognitive scaffolding is smaller than it was two model generations ago. That is real, and it is enough to motivate the audit cadence proposed below. But it is not the strong directional claim that "complexity should ratchet down."

Migration, not deletion: the Claude Code TodoWrite case

The most candid published account of scaffolding becoming counterproductive comes from Anthropic's Claude Code design retrospective. The team describes introducing a TodoWrite tool plus reminders to keep early Claude Code on-task across long workflows. As underlying models improved, the same mechanism started causing problems: the model would over-anchor to plans it had written down earlier, even when partial progress, new information, or user feedback should have prompted revision. The scaffolding was solving a problem (task drift) that the current model no longer had, while creating a new problem (plan ossification) that the current model was uniquely susceptible to.

Two things about the response are instructive.

First, the response was not deletion. The team replaced TodoWrite with a Task tool that handled dependencies and subagent coordination, added an AskUserQuestion tool to surface ambiguity instead of plowing through it, added grep and codebase-search primitives, added skills as a way to encode reusable workflows, and added a Claude Code Guide subagent to keep the main context clean. The complexity that had lived in prompt-level reminders moved to interface design, tool design, and subagent decomposition. By any reasonable accounting, the system grew more sophisticated, not less. What changed was where the sophistication lived.

Second, the same retrospective is explicit that prior tool assumptions should be revisited at each major model upgrade. The implied audit cadence is "model release," not "every quarter" or "every postmortem." This is consistent with the pruning principle's strong intuition — that model capability changes the cost-benefit of every component built around the model — without endorsing a pure-deletion response.

The right generalization from this case is that scaffolding which shapes the model's cognition (TodoWrite-style reminders, manual planners, structured-thought preambles) is genuinely vulnerable to obsolescence as models improve, because it overlaps directly with capabilities the model is gaining. Scaffolding which expands the model's contact with reality (better tools, cleaner subagent boundaries, sharper user-clarification flows) tends to compound with model improvement rather than compete with it.

This is the architectural inversion that matters: as models improve, complexity should migrate outward — from prompt-level cognition shaping to environmental affordances and verification — rather than disappearing. The strongest version of the pruning principle is partly a misreading of this migration as net deletion.

What "dead weight" actually looks like

Some scaffolding really is just dead. The honest empirical case for the pruning principle rests on a small number of patterns where the diagnosis is reasonably clear and the removal is reasonably testable. Each of these has a documented basis in primary guidance and an empirical signature you can detect.

Manual chain-of-thought prompts on reasoning models

OpenAI's reasoning model best practices state directly that reasoning models perform best with concise, well-specified prompts and that "think step by step" preambles often hurt rather than help. The mechanism is straightforward: the model already runs an internal reasoning trace; an external request to think step-by-step either duplicates that trace, distorts it by adding format constraints, or pushes the model into a less optimal reasoning mode. The empirical signature is task accuracy and cost on a paired ablation: same prompts with and without the CoT preamble, on a held-out task set. If accuracy is equal or worse and tokens are higher with the preamble, it is dead weight.

This is one of the few anti-patterns where the diagnosis, the mechanism, the official guidance, and the test are all clean.

Output-format coercion that duplicates structured outputs

Many production prompts spend tokens explaining a JSON schema in natural language, providing examples, and instructing the model to "respond with valid JSON." When the underlying API supports structured outputs natively (constrained decoding against a schema), the prompt-level coercion adds tokens without adding correctness. The signature is failure-rate parity: structured-output mode at zero schema-violation rate makes the prose instructions vestigial. The cost is small per call and large at scale.

Wrappers that re-implement primitives the model now invokes well

Early agent designs frequently included hand-written planners that decomposed a user request into subgoals, then invoked the model on each subgoal. For sufficiently capable models, the planner adds a round trip and constrains the model's strategy without adding accuracy. The signature is: replace the planner with a single call that gives the model the same tools and asks for the same output, on a controlled task set, and measure delta. If success rate and quality are equal or better and total tokens are lower, the planner is dead weight. This is the case the mini-SWE-agent result most cleanly supports — for at least one benchmark and a current frontier model, the framework's planner layer was not earning its keep against a single tight loop.

Pre-injected RAG context for queries the model can search itself

Retrieval-augmented generation pipelines that pre-load context into the prompt — regardless of whether the query needs it — add tokens, latency, and the risk of context pollution from stale documents. When the model has competent search tools and can issue retrievals on demand, eager RAG often loses to lazy retrieval. The signature is: same-task accuracy with eager RAG versus tools-and-search, holding context budget constant. Eager retrieval still wins for very short queries with a known relevant corpus and for cases where retrieval latency dominates. Outside those cases, the architecture is increasingly suspect.

Redundant safety preprompts that duplicate system-level enforcement

A safety preprompt that says "do not produce harmful content" inside a chat where the API-level safety system already enforces the same policy adds nothing except tokens and the occasional false-positive refusal cascade. This is genuinely dead weight, and its removal is straightforward — provided it is clear that the duplication is real, that the policy is identical, and that no audit requirement depends on the visible string. (See the next section for why that "provided" matters.)

Output validators that re-check what the model produced from a constrained schema

When a model is producing output via constrained decoding against a schema, an additional Python-side schema validator that throws on mismatch is testing for an event that cannot occur. The validator may have been added when the model produced free-form JSON; if the API has since moved to structured outputs, the validator has no failure mode left to catch.

What unifies these patterns is a clean diagnostic: a controlled test in which the scaffolding is removed should produce equal or better outcomes on the metrics that originally motivated it, with no new safety, observability, or recovery cost. Without that test, "looks like dead weight" is taste, not engineering.

What is not dead weight

The most consequential failure mode in writing about pruning is to conflate the patterns above with operational scaffolding that has nothing to do with model capability. When the audit criterion is "feels heavy" rather than "fails its diagnostic test," entire categories of useful infrastructure get caught in the sweep.

Permission systems and approval policies

Codex's approvals system and Claude Code's analogous permission modes exist because execution authority is a deployment concern, not a model-quality concern. A more capable model can write better commands and also execute more dangerous ones faster. The argument for approval gates does not weaken with model improvement; for irreversible actions, it strengthens. Removing approval requirements because "the model is smart enough now" misreads the failure mode being prevented. The point of approval is not "catch a confused model"; it is "create a checkpoint at which the user or an audit process can intervene before damage is final."

Sandboxing and blast-radius controls

Sandboxed execution, per-command capability limits, network egress restrictions, and write-access gates all exist to bound the consequences of any action — correct or incorrect — that an agent takes. These controls assume the agent will sometimes be wrong, sometimes be jailbroken, and sometimes be entirely correct but acting on incorrect human input. None of these failure modes is solved by model improvement.

Audit logs and observability

Recording what an agent did, with what arguments, against what state, and with what outcome serves three independent purposes: incident investigation, regulatory compliance, and continual evaluation. None of these depends on model quality. A more capable model produces more interesting traces, not fewer.

Verification harnesses

Test execution against a known-good suite, golden-output diffs, judge-model evaluations, and structural validators against typed contracts catch errors that the model cannot self-detect without additional capability the verification provides. Some specific verification steps become unnecessary as models improve (a model that always produces valid JSON makes a JSON validator vestigial), but verification as a layer persists because its job is to confirm reality, not to compensate for cognitive limits.

Domain knowledge in AGENTS.md, skills, and durable instructions

OpenAI's Codex best practices frame Codex as a teammate that is configured over time, with AGENTS.md carrying durable project-level guidance, MCP servers for specialized capabilities, skills for reusable workflows, and a deliberate practice of feeding repeated mistakes back into the durable instructions. This is scaffold accumulation under discipline. The same document warns against vague or bloated guidance — the discipline matters — but the accumulation is the point. Domain knowledge that took a senior engineer six months to learn does not become unnecessary when the model gets better; it becomes more accessible to the model, which is a different thing.

Context management

Subagents, summarization layers, and context-window economics tools do not exist because the model cannot reason; they exist because context windows have economic structure and because long-running agents accumulate state that is expensive to keep visible. Codex's subagents documentation frames them as a tool for handling context pollution and parallel exploration with explicit cost trade-offs. Better models do not change the underlying constraint that attention over a million tokens costs more than attention over ten thousand.

The principle here is that scaffolding which manages the world the model operates in — its execution authority, its action consequences, its evidence base, its memory, and its institutional knowledge — is governed by forces orthogonal to model capability. Treating it as dead weight is the central failure mode of an over-strong pruning thesis.

Recognizing ossified scaffolding

A useful audit needs falsification criteria, not aesthetic preferences. The following diagnostic tests separate scaffolding that is genuinely vulnerable to model improvement from scaffolding that is performing a different job.

The "what failure does this prevent?" test

For each scaffolding component, ask: what specific failure mode was this introduced to address? If the answer is in the cognitive-shaping category — task drift, format violations, planning incoherence, reasoning gaps, instruction-following lapses — the component is in the audit zone. If the answer is in the operational category — unauthorized actions, untraceable executions, irreversible mistakes, regulatory violations, blast-radius exceedance — the component is doing a job that model improvement does not retire.

Components whose original justification has been forgotten are not necessarily dead weight; they are more often institutional knowledge that has not been written down. Pruning before recovering the justification is an error.

The paired-ablation test

Run the system with and without the component on a representative task set, holding everything else constant. The component is earning its keep if removing it produces measurable degradation on the metric it was introduced to defend (accuracy, latency, cost, safety incident rate, recoverability). The component is dead weight if removing it produces no degradation or improves the metric.

This is the only audit standard that can override taste. The mini-SWE-agent result is most cleanly read as a paired ablation: same model, comparable tasks, scaffold-light versus scaffold-heavy, and the scaffold-heavy version no longer wins on the SWE-bench task distribution.

A paired ablation needs three things to be useful:

  • A representative task distribution. SWE-bench is one distribution. A production system's actual task mix is usually narrower or wider in different ways. An ablation against your real workload tells you about your real workload.
  • A budget cap that matches production. Many "framework wins" disappear at fixed token budgets, and many "minimal-scaffold wins" disappear if the framework version is allowed to retry until success. Match the budget you actually deploy with.
  • Metrics that include safety, recoverability, and observability. A scaffold whose removal saves 15% of tokens but introduces a new class of unrecoverable failure is not a win. The ablation must look at the full failure surface.

The "what does the model now do natively?" test

When evaluating a cognitive scaffold, check whether the underlying model now does the same thing internally. Reasoning models that internally produce a chain of thought have made manual CoT prompts redundant. Models that natively support tool use have made hand-coded ReAct loops redundant. Models that produce constrained-schema output have made format-coercion prompts redundant. The test is: does the model produce the scaffolded behavior even when the scaffold is absent? If yes, the scaffold is duplicating the model's own work.

This test is sharper than a general "is the model better now" question because it focuses on overlap with a specific model capability rather than overall capability.

The "did this survive a model release?" question

If a scaffold component was added two model generations ago, never reviewed, and is still in the prompt or the orchestration layer, the prior probability that it earns its place today is lower than it was the day it was added. Not zero — many components are model-independent — but lower. This is a heuristic for where to look, not a conclusion. Long-lived scaffolding deserves audit not because age is bad but because the conditions that produced it have almost certainly changed.

The "context-pollution" check

Some scaffolding does not fail by being wrong; it fails by being present. A long preamble that crowds out task-relevant context, a tool spec that sits in every turn whether or not the tool gets called, a system prompt that has accumulated three years of conditional clauses — these can degrade performance even when each individual addition was correct in isolation. The signature is that the model performs worse on tasks that require deep attention to user input than the model's underlying capability would predict. The fix is not always pruning; sometimes it is moving the content to a place the model can read on demand (a skill, a tool description loaded only when invoked, a retrievable document) rather than carrying it in every prompt.

Anti-patterns with diagnostics

The following anti-patterns are commonly cited in the pruning-principle literature. For each one, the table below pairs the symptom with the test that would actually establish whether the pattern is dead weight.

Anti-pattern Symptom Diagnostic test Removal experiment
Manual CoT preamble on reasoning models Reasoning model receives "think step by step" or structured-thought prompt Paired ablation on accuracy and tokens for held-out tasks Remove preamble; measure accuracy delta and token delta; deploy if non-negative on accuracy and negative on cost
Hand-written planner around a tool-using model Planner decomposes task into subgoals, then invokes model per subgoal Single-call version with same tools and same task on same model Replace with single call; measure success rate, quality (blind eval), and total tokens
Output-format coercion with structured outputs available Prompt explains JSON schema in prose; model occasionally produces invalid JSON anyway Switch to constrained decoding; measure schema-violation rate (should drop to 0) and prompt-token savings Move schema to API-level structured output; remove prose coercion
Eager RAG when model has search tools Pre-loaded retrieved context appears in every prompt Lazy-retrieval version: tools available, no preload Compare accuracy and cost on real query distribution; preserve eager only for very short queries with known small corpus
Redundant safety preprompts System prompt restates platform-level safety policy verbatim Audit whether removal changes refusal rate, harmful-output rate, or compliance evidence Remove; verify behavior parity on red-team eval; check that audit/compliance does not depend on visible string
Stale persistent reminders or to-do scaffolds Model anchors to old plan, fails to revise on new information Inspect a sample of failed runs for plan-ossification signature Replace reminder with on-demand recall (Task tool, query-when-needed) rather than persistent injection
Output validator after constrained decoding Validator never throws Log validator invocations and outcomes for one week Remove if violation rate is zero; keep if it catches non-schema failures (e.g., logical constraints)
Subagent for trivially simple work Subagent invocation overhead exceeds the task it performs Measure end-to-end latency and token cost with and without subagent layer Inline the work for sub-threshold tasks; preserve subagent for genuine context-isolation needs
MCP server for a capability the model has natively MCP tool wraps something the model could do without external help Comparison of task success with tool available vs. without Remove if equivalent; preserve if it adds reliability, auditability, or determinism

The structure of each diagnostic is the same: name the failure mode the scaffold prevents, design a test in which the failure mode could surface if the scaffold is removed, and set a removal threshold (typically: equal-or-better on the defended metric, no new failure modes introduced). Without a diagnostic, "this looks bloated" is not a sufficient reason to remove it.

Audit cadence and triggers

The right cadence for scaffold audit is event-driven, not calendar-driven. Periodic reviews tend to be either too frequent (reviewing scaffolds that have not had their context change) or too infrequent (missing the moment when a model release made something obsolete). Trigger-driven audits target the moments when the cost-benefit calculus actually moved.

The triggers that warrant audit:

  • Model release within the deployed family. A new generation of the underlying model is the single highest-information event for scaffold audit. Plan an audit pass within the week after upgrading. The Claude Code TodoWrite case is the canonical example — the scaffolding became counterproductive at a specific model upgrade.
  • Model swap to a different family. Switching from one model family to another (Claude to GPT, GPT to Gemini) changes capability profile, instruction-following style, and tool-use behavior; many scaffolds tuned to one family work poorly with another.
  • Reasoning-mode change. Turning reasoning effort up or down, or switching from a non-reasoning to a reasoning model, can make manual CoT scaffolds harmful where they were previously helpful.
  • A failure cluster surfaces in postmortems. Three or more failures with a common root cause are evidence that some scaffold either is missing or is not doing its job. The audit question is which.
  • Latency or cost regression in production telemetry. A 30% increase in tokens-per-task without an accompanying improvement in success rate is a signal that some recently-added or recently-grown component is not earning its rent.
  • Benchmark methodology change. When a benchmark you have been optimizing against changes its task distribution, scoring, or contamination posture, scaffolds tuned to the old benchmark may now be optimizing for the wrong target.
  • Major prompt or tool refactor. Whenever you are touching the orchestration layer for unrelated reasons, a focused audit of nearby scaffolds is cheap and timely.
  • Incident. A safety, security, or correctness incident is always an occasion for scaffold audit — both to check whether removable scaffolding was masking the failure and to check whether non-removable scaffolding (permissions, audit) needs to be strengthened.

The cadence should produce a written record. Without a record of "this scaffold was reviewed at this model upgrade and retained because of this measured benefit," you cannot distinguish institutional knowledge from sunk-cost preservation, and the scaffold's existence will be relitigated every time someone new looks at it.

Why scaffolding ossifies

A purely technical account of scaffold ossification — "scaffolding hangs around because nobody removed it" — misses the organizational structure that keeps it in place. The same scaffold can be technically obsolete and operationally indispensable for reasons that have nothing to do with model capability.

Sunk cost and authorship attachment. Someone built it. Someone defended it in a review. Removing it implicitly says that work was wrong. In well-functioning teams this is overcome by paired-ablation evidence. In less-functional teams it is not.

Compliance and audit dependency. A scaffold may have been added to satisfy an external auditor, a regulatory requirement, or a contract clause. Its removal, even when it is technically dead weight, requires re-engaging the requirement that originally produced it. Many "dead" scaffolds are actually living artifacts of compliance regimes that nobody wants to reopen.

Ownership boundaries. A scaffold owned by Team A may serve Team B and C, who do not have authority to remove it. The audit signal exists; the audit response is blocked by the org chart.

Eval inertia. Scaffolds tuned to a specific eval suite tend to accumulate around the eval's idiosyncrasies. Removing them risks regressing on the eval, even if the regression is itself an artifact of the eval's narrowness rather than a real capability loss. Without confidence that the eval reflects production, teams retain scaffolds defensively.

Vendor lock-in and tooling assumptions. Scaffolds that integrate with specific vendor tools (a particular tracing system, a particular framework's middleware contract, a particular orchestration platform) become harder to remove than the scaffold's intrinsic complexity would suggest. Removing one scaffold can require migrating the surrounding tooling.

Risk asymmetry. Removing a scaffold that turns out to have been load-bearing is a visible failure. Retaining a scaffold that turns out to be dead weight is an invisible cost. Most production environments have asymmetric incentives that favor over-retention.

The pruning principle works in environments where these forces are weak — small teams, internal tools, well-instrumented evaluations, low compliance burden. It works less well in environments where they are strong — large organizations, regulated industries, multi-team systems, weak observability. The principle is not wrong in those environments; the audit threshold for removal is just much higher, and the right answer is often migration (per the Claude Code retrospective) rather than deletion.

The corrected principle

The original framing — "scaffold complexity should ratchet down, not up, with model improvement" — does not survive contact with the evidence. It overgeneralizes from a narrow benchmark result, conflates distinct categories of scaffolding under one prescription, and ignores the role of operational, safety, and organizational concerns that model improvement does not address.

The version that does survive is narrower and stricter:

Cognitive scaffolding that compensates for model limitations should be aggressively re-audited at every model release. Where a paired ablation shows no current benefit on the metrics the scaffold was introduced to defend, the scaffold should be removed. Where the same audit shows that the scaffold is now actively harming performance — by polluting context, anchoring stale plans, distorting reasoning, or duplicating native model capability — its removal should be a priority, not a low-stakes cleanup. Scaffolding that handles environment, verification, safety, observability, or organizational coordination operates under different forces and should be evaluated on its own terms, not under the same prescription.

Two consequences follow from this restatement.

First, the architectural trajectory of well-designed agent systems is not toward less complexity overall but toward complexity in different layers. As models gain native reasoning, planning, format adherence, and instruction following, the prompt layer thins. As the prompt layer thins, complexity should — and historically does — accumulate in environmental affordances (better tools, cleaner subagent boundaries, sharper handoffs), in verification (richer evals, judge models, retry-with-critique on the right axes), and in organizational scaffolding (durable instructions, learned-from-failure patterns, project-specific skills). The Claude Code design story is the canonical demonstration: deletion at the prompt layer, accumulation at the tool and subagent layer.

Second, the audit cadence that the strong pruning principle implicitly recommends — review on model release, design ablations to test removal, write down the justification for each retained component — is good advice independent of whether you accept the strong version. Even if you believe scaffolding rarely goes away, the discipline of forcing every component to re-justify itself at each model upgrade is the only protection against ossification. The audit is the load-bearing recommendation. The directional claim about ratcheting down is the disposable rhetoric around it.

A reader of the broader pruning literature should treat the slogan version of the principle the way one would treat any strong directional claim about software architecture: as a useful intuition pump for one specific failure mode (cognitive scaffolds outliving the model limitations they compensated for) that does not generalize to a complete philosophy of agent design. The audit cadence is the engineering. The slogan is the marketing.

What would change this assessment

Several near-term developments could shift the picture toward the stronger version of the pruning principle:

  • Clean same-model paired ablations across multiple tasks. The strongest current evidence is leaderboard convergence on one benchmark, which confounds many variables. Public reports of paired ablations — same model, varied scaffolds, multiple task families, fixed budget — would either harden the case for pruning or sharply qualify it.
  • A sustained pattern of model releases that retire specific scaffold categories. Two or three more iterations of the TodoWrite pattern, where Anthropic, OpenAI, or another major lab publishes that a specific category of tooling became counterproductive at a specific release, would establish that this is a recurring phenomenon rather than an isolated case.
  • Native model capabilities that subsume currently-load-bearing tools. If models develop reliable long-horizon planning that does not require external scheduler subagents, or reliable multi-document retrieval that does not require eager RAG, the categories of scaffolding genuinely vulnerable to obsolescence would expand.
  • Framework benchmarks that include safety, recoverability, and audit metrics. Most current public benchmarks measure task success and cost. A benchmark that scored agents on incident rate, recoverability from bad state, and audit completeness would surface the value of operational scaffolding that current benchmarks ignore.

Conversely, several developments would weaken the pruning principle further:

  • A new model generation where minimal scaffolds underperform framework-heavy ones. Capability progress is not monotonic across all dimensions; a model release that regresses on an axis previously absorbed by minimal scaffolds would force the reintroduction of compensating structure.
  • A high-profile incident where scaffolding removal caused a safety, security, or correctness failure. Incidents have outsized influence on subsequent design conservatism.
  • Stronger evidence that current benchmark convergence is benchmark-specific. If mini-SWE-agent's performance on SWE-bench Verified turns out to be a benchmark artifact rather than a generalizable claim about coding agents, the central empirical anchor for the principle weakens.

The honest position is that the pruning principle in its strong form is undersupported, in its weak form is well-supported, and in its operational form (audit cadence, paired ablations, category-aware analysis) is a solid engineering discipline regardless of which way the directional claim eventually settles.

Companion entries

Core theory:

  • Building Effective Agents
  • Agent-Computer Interface Design
  • Scaffolding Taxonomy: Cognitive vs. Environmental vs. Operational
  • Cognitive Scaffolding and Model Capability Overlap

Practice:

Empirical evidence:

  • SWE-bench Verified: Methodology and Limitations
  • Mini-SWE-Agent and the Bash-Only Baseline
  • The Original SWE-Agent Paper: Why Interface Design Mattered
  • OpenAI's Withdrawal from SWE-bench Verified Evaluation
  • Reasoning Model Prompting: What Helps and What Hurts

Counterarguments:

  • Why Safety Scaffolding Does Not Prune
  • Permissions and Approval Policies for Autonomous Agents
  • Observability as Independent Concern
  • Sunk Cost and Compliance Inertia in Scaffold Removal
  • Migration vs. Deletion: The TodoWrite Case

Adjacent domains:

  • Verification Loops and Evaluator-Optimizer Patterns
  • Sandbox Design for Code-Executing Agents
  • Tool-Use Patterns Across Model Families
  • Benchmark Saturation and Architectural Inference

AI-researched reference article. Something wrong here? Tell us.