The Economics of Coding Agents in Legacy Systems

22 min read
The Economics of Coding Agents in Legacy Systems

In greenfield prototypes, the economics of coding agents look deceptively linear: prompt a model, generate hundreds of lines of scaffolding, run a happy-path check, and ship. Because there are no existing callers to break, no historical data migrations to respect, and no live transaction volume at risk, the cost of a change is dominated almost entirely by how fast the model can emit tokens.

In a mature, revenue-generating system, those conditions invert. Legacy systems are defined not by the age of their syntax, but by the density of their implicit invariants. Every module encodes years of operational scar tissue: idempotency boundaries, lock acquisition orderings, partner retry quirks, and regulatory audit trails that rarely live in a single README. When you loose an unconstrained coding agent on a codebase like this, making code generation cheap does not reduce engineering costs. Instead, it shifts those costs downstream into architectural coupling, senior review bottlenecks, context rediscovery, rework, and production incidents.

Treat coding agents as a stochastic subsystem with quantifiable downstream externalities: cheap tokens can quietly manufacture expensive review queues, rework, and production risk.

1. The Core Unit of Account: Total Cost per Accepted Change

Engineering organizations routinely evaluate AI coding tools using proxy metrics: tokens per diff, lines of code generated per minute, or percentage of PRs touched by an agent. As Donald Reinertsen shows in The Principles of Product Development Flow (Pt. 1), optimizing a local proxy variable while ignoring downstream holding and verification costs actively degrades total system throughput. Generating an unverified 800-line diff in thirty seconds is not productivity; it is the rapid creation of unverified inventory.

Economic Trade-Off: Authoring Cost vs. Complexity Cost

Authoring Cost (Prompt-to-Diff) Complexity & Coupling Cost Total Economic Cost Optimal: Bounded Harness Step Unconstrained Loop (Coupling Spike)

Figure 1: Agent autonomy follows a U-curve optimization between declining Authoring Cost and rising Complexity & Coupling Cost, reaching minimum total cost at small, bounded steps.

Where a task lands on this curve depends on four things:

  1. Files and shared helpers touched: Whether edits stay within a single module or alter shared utilities imported by sibling services.
  2. State transitions modified: Whether the diff changes database schemas, lock acquisition order, or asynchronous retry state.
  3. Invariants affected: Whether idempotency, ledger balancing, or external partner contract assumptions are in scope.
  4. Expected review effort: The senior engineering time required to verify that every touched path preserves production behavior.

The only honest unit of account in software delivery is the Total Cost per Accepted Change, which is the full economic cost required to take a requirement from prompt to a merged, production-safe commit that does not regress system invariants:

Cost per Accepted Change=Cgen+Crediscovery+Creview+Crework+Cdelay+(p×L)\text{Cost per Accepted Change} = C_{\text{gen}} + C_{\text{rediscovery}} + C_{\text{review}} + C_{\text{rework}} + C_{\text{delay}} + (p \times L)

Where:

  • CgenC_{\text{gen}} (Generation Cost): The direct inference cost of model tokens consumed to produce the candidate change.
  • CrediscoveryC_{\text{rediscovery}} (Rediscovery Tax): The token and compute cost of the agent repeatedly searching, grepping, reading, and compacting the repository to reconstruct architectural context it does not natively possess.
  • CreviewC_{\text{review}} (Senior Review Cost): The fully loaded engineering time required for a human domain expert to audit the diff, trace cross-module side effects, and verify that hidden invariants still hold.
  • CreworkC_{\text{rework}} (Rework Cost): The engineering and compute time spent fixing broken callers, subtle regressions, or failed CI runs after the initial diff is generated.
  • CdelayC_{\text{delay}} (Queue Delay Cost): The cost of delay incurred while a sprawling, high-risk pull request sits waiting for a senior engineer with enough uninterrupted calendar time to review it.
  • p×Lp \times L (Expected Incident Cost): The probability (pp) that the change introduces a latent production defect multiplied by the expected financial and operational loss (LL) of the resulting incident.

When we compare an unconstrained repo-wide agent loop against a bounded harness step across these six variables (assuming an illustrative $150/hr loaded senior engineering rate and a $20,000 incident cost when a settlement invariant breaks), the unit economics come into sharp focus:

Cost Component Unconstrained Repo-Wide Loop Bounded Harness Step What Drives the Difference
Generation (CgenC_{\text{gen}}) $6 to $15 $2 to $5 Unbounded multi-turn token sprawl vs. scoped model tokens
Rediscovery (CrediscoveryC_{\text{rediscovery}}) $10 to $25 $1 to $3 Repeated repo searches & lossy compaction vs. pre-indexed graph
Senior Review (CreviewC_{\text{review}}) $225 (90 min) $50 (20 min) Auditing cross-module side effects vs. verifying a bounded contract
Rework (CreworkC_{\text{rework}}) $150 (60 min) $25 (10 min) Fixing broken callers/tests vs. minor revisions (bad runs gated)
Queue Delay (CdelayC_{\text{delay}}) $300 (multi-day wait) $40 (same-day) Large diffs stalling in the review bottleneck vs. small-batch flow
Expected Incident (p×Lp \times L) $400 (2.0% × $20k) $40 (0.2% × $20k) Unchecked cross-module mutations vs. contract-locked boundaries
Illustrative Total ~$1,095 / change ~$160 / change Raw tokens are <2% of the unconstrained cost

Notice what the breakdown reveals: raw inference tokens (CgenC_{\text{gen}}) account for less than 2% of the total cost of an unconstrained change. Even when a larger cross-cutting requirement must be decomposed into three sequential bounded harness steps (3×$160=$4803 \times \$160 = \$480, plus ~$30 in amortized contract setup and graph indexing), the decomposed workflow still cuts expected delivery cost by more than half compared to a single unconstrained repo-wide loop (~$1,095), primarily by compressing review stalls and tail-risk exposure under these assumptions.

Observational pull-request telemetry from our own workflows reflects this divergence. Across 142 agent-assisted pull requests tracked over a 60-day window in a transactional codebase, changes that were decomposed and gated inside a single-module harness step (1 to 2 files, n=89n = 89) waited nearly ten times less for initial human review (3.2 hours vs. 31.5 hours, ~9.8×) and required roughly one-fourth the active review time (18 minutes vs. 74 minutes, ~4.1×) compared with unconstrained runs allowed to touch 5+ files and shared helpers (n=53n = 53):

Observed PR Cohort (60-Day Sample, N=142N = 142) Scoped Harness (1 to 2 files, single module, n=89n = 89) Unbounded Loop (5+ files, shared helpers touched, n=53n = 53)
Median Queue Wait Before First Review 3.2 hours 31.5 hours
Median Senior Review Duration 18 minutes 74 minutes
Review Rounds to Merge 1.1 rounds 3.4 rounds
Pre-Merge Discard or Reset Rate 7.9% (7 / 89, caught by harness gates) 37.7% (20 / 53, abandoned after review pushback)
Post-Merge Hotfix or Rollback Rate 0% (0 / 82 merged) 6.1% (2 / 33 merged)

Note: Cohort metrics are observational rather than randomized; because the harness enforces decomposition of larger tasks into 1-to-2 file steps before code generation, the improvements reflect the combined effect of smaller batch size and automated pre-review gating. Review duration measures active reviewer session time; rollback rates track 14-day post-merge regressions.

2. Programming With a Loom: Easy vs. Simple in Legacy Code

In his classic talk Simple Made Easy, Rich Hickey draws a sharp distinction between easy (what is near at hand, familiar, or low-friction to produce) and simple (what is unbraided, derived from the Latin simplex, meaning “one fold,” as opposed to complect, meaning “to weave or braid together”).

Coding agents make code generation frictionless. They make authoring easy. Left unconstrained in a legacy repository, however, an agent acts like a high-speed mechanical loom: it solves immediate local compiler errors or feature prompts by braiding state, imports, and side effects across module boundaries.

The Anchor Case: settleBatch in a Double-Entry Ledger

To see how this braiding happens in practice, and to trace how a single unconstrained edit cascades through every variable in our cost equation, consider a concrete production routine: settleBatch inside a payment and settlement service. In the existing path, settleBatch enforces two critical invariants inside a database transaction:

  1. Invariant 1 (Lexicographical Lock Ordering): Acquire row-level locks on merchant and partner accounts in strict lexicographical order by account_id to prevent deadlocks under concurrent webhook and payout workers.
  2. Invariant 2 (Double-Entry Conservation): Double-entry postings constructed by buildDoubleEntryPostings(entries) must foot strictly to zero (totalDebits === totalCredits) before inserting ledger rows.

Suppose we ask an agent to add partner fee withholding to settleBatch. An unconstrained agent with full write access to the repository frequently exhibits two destructive behaviors:

  1. Silent Invariant Erosion: To deduct the partner fee quickly, the agent fetches and locks the partner fee account inside a helper function after the main entries loop has already locked a subset of merchant accounts, and mutates a shared posting helper imported by a dozen sibling handlers. The code compiles and passes single-threaded unit tests, but it violates the strict lexicographical account_id lock ordering, planting a latent database deadlock that will only detonate under peak concurrent settlement load.
  2. Guard Relaxation and Assertion Weakening: When the naive fee deduction causes totalDebits to exceed totalCredits, settleBatch throws UnbalancedLedgerError and fails the characterization test (settleBatch.spec.ts). To force a green build, the unconstrained agent relaxes the runtime check in settleBatch.ts from totalDebits !== totalCredits to totalDebits < totalCredits and rewrites the test assertion from strict equality to toBeGreaterThanOrEqual(receipt.totalCredits).

The agent has optimized for its immediate reward signal, a passing test command, by destroying the single most important financial invariant in the codebase. A senior reviewer must now spend ninety minutes tracing callers of the shared helper and reconstructing lock-acquisition order across the repository.

A bounded harness blocks this reward-hacking path before a PR is opened. By restricting edits to settlement/settleBatch.ts and settlement/postings.ts, locking settlement/settleBatch.spec.ts as read-only, and running a property-based zero-sum invariant gate (totalDebits === totalCredits), the host rejects any attempt to weaken the test or bypass double-entry conservation. Forced to satisfy the invariant, the agent implements proper net fee withholding on the primary merchant payout entry: splitting that gross settlement entry into a net merchant credit (primary.amount - feeAmount) and an offsetting partner-fee credit (feeAmount) against the original settlement debit (while preserving any remaining batch entries) before account identifiers are sorted and locked. The resulting diff touches two domain files, preserves global lock ordering, and can be verified by a senior engineer in fifteen minutes.

Shifting Verification Left: Deterministic Gates Inside the Harness

Bounding the edit surface reduces diff size, but compressing CreviewC_{\text{review}} and incident probability (p×Lp \times L) requires shifting verification from human visual inspection to deterministic machine checks inside the harness loop. Standard unit tests are insufficient because an unconstrained agent can either modify the tests or satisfy narrow fixture inputs while violating global system invariants.

Effective legacy harnesses enforce three automated verification layers before a pull request ever enters a human review queue:

  1. Architectural boundary linters: Static checks that fail the run if a diff modifies exported signatures in shared packages, introduces unauthorized cross-service imports, or invokes database transaction primitives outside designated repository layers.
  2. Stateful property-based invariant checks: Rather than testing a single hardcoded payload, the harness runs randomized sequences of concurrent requests, retries, and partial reversals against the modified module, asserting three universal invariants on every run:
    • Zero-sum conservation: Across every transaction boundary, total debits equal total credits.
    • Idempotency convergence: Replaying the same request or webhook payload multiple times under concurrent workers yields the exact same terminal ledger state and side-effect count as executing it once.
    • Monotonic state transitions: Accounts and settlement batches never transition backward from terminal states such as settled or reversed back to pending.
  3. Immutable test and contract locks: Existing characterization tests and API schemas are mounted read-only during implementation. When a change breaks an existing test, the agent must fix its implementation rather than rewriting the assertion to match its bug.

3. Queueing Theory and Classical Systems Laws

Why does an edit like the settleBatch refactor blow up engineering economics even when a human reviewer eventually catches the bug before merge? The answer lies in operations research and three classical laws of systems engineering.

The Verification Bottleneck and the 20-Minute Calendar Threshold

As detailed in Principles of Product Development Flow (Pt. 2), in any system with variable arrivals and non-zero service times, queues form immediately upstream of the most constrained resource. Coding agents increase the arrival rate of candidate diffs by an order of magnitude while leaving the true bottleneck, senior human verification capacity, completely unchanged.

By Kingman’s formula, as utilization (ρ\rho) of senior reviewers approaches 100%, wait time in the review queue (WqW_q) grows non-linearly with arrival variability (ca2c_a^2), service-time variability (cs2c_s^2), and mean review duration (τ\tau):

Wq≈(ρ1−ρ)⋅(ca2+cs22)⋅τW_q \approx \left(\frac{\rho}{1 - \rho}\right) \cdot \left(\frac{c_a^2 + c_s^2}{2}\right) \cdot \tau

In our experience on engineering teams, service time (τ\tau) and its variability (cs2c_s^2) often exhibit a practical scheduling discontinuity around a familiar human heuristic: the 20-minute calendar gap between meetings.

  • Below the threshold (1 to 2 files, bounded contract, locked tests): If a change to settleBatch is restricted to a single module and verified by an immutable test suite, a staff engineer can review and approve it in 10 to 20 minutes between meetings. Batch flow stays continuous, and CdelayC_{\text{delay}} remains low (~$40).
  • Above the threshold (5 to 7+ files, cross-module helper edits, touched test files): The moment the agent modifies settleBatch, alters a shared lock helper, updates three upstream callers, and touches settleBatch.spec.ts, the diff crosses the calendar threshold. The reviewer can no longer trust the test suite without auditing the test changes themselves, and must manually trace lexicographical lock ordering across seven files. That requires a 90-minute block of deep, uninterrupted focus (Creview=$225C_{\text{review}} = \$225). Because senior engineers rarely have spare 90-minute blocks mid-week, the PR stalls in the review queue for days (Cdelay=$300C_{\text{delay}} = \$300), blocking dependent work and compounding merge drift.

The Queue Multiplier Effect

Queue Verification Capacity Utilisation 80% 1× queue 90% 2× doubles 95% 4× doubles again 100%

Cumulative Flow: Widening WIP vs. Stable Flow

Unconstrained Loop Widening WIP Arrival > safe review capacity Governed Harness Parallel flow Thin WIP · short cycle time Band height = unverified diffs in review

Figure 2: Pushing verification capacity toward 90–95%+ utilization triggers the hyperbolic (nonlinear) Queue Multiplier Effect (left) and widens the Unverified WIP band on the Cumulative Flow Diagram (right).

To keep arrival and departure lines parallel on the Cumulative Flow Diagram, teams need explicit operating policies around the review bottleneck:

  • Hard WIP limits per reviewer: Cap the number of active, unmerged agent PRs assigned to any senior reviewer (for example, at most 2 concurrent diffs) so review queues cannot grow silently.
  • Maximum review-queue age: Track the age of unverified diffs; if review wait time crosses a defined SLA (such as 24 to 48 hours), pause new automated diff generation until the queue clears.
  • Stop-generation rule on CFD divergence: When the arrival rate of agent-generated PRs exceeds the departure rate of verified merges, treat the widening WIP band as a backpressure signal to throttle task dispatch.
  • Risk-weighted sequencing over FIFO: Because software changes differ sharply in cost of delay and blast radius, review queues should never be processed first-in, first-out. Sequence small, isolated fixes with deterministic test evidence ahead of cross-service changes, and schedule ledger or state-machine modifications into dedicated architectural review blocks.

Incremental Evolution vs. Clean-Slate Rewrites (Gall’s Law)

John Gall’s observation in Systemantics, that every complex system that works evolved from a simpler system that worked, applies directly to agentic refactoring. When an agent encounters tangled legacy control flow around settleBatch, teams are often tempted to let the model rewrite the settlement module cleanly from scratch.

Clean-slate agent rewrites fail in legacy systems because the existing control flow is the specification of historical edge cases. Unless you first extract and lock an explicit invariant inventory, a rewrite silently drops load-bearing behaviors:

Invariant Class in settleBatch Where It Lives in Legacy Code What Happens in an Unbounded Agent Rewrite
Concurrency & Lock Order Lexicographical .sort() on account_id before SELECT ... FOR UPDATE Replaced with ad-hoc per-account locking inside loops; causes production deadlocks.
Double-Entry Conservation Strict totalDebits === totalCredits check across all legs Weakened or bypassed when adding fee/tax legs without offsetting balancing entries.
Idempotency & Replay Permanent claim row keyed on (scope, idempotency_key) Simplified into an in-memory or short-TTL check; allows duplicate credits on webhook replay.
Partner Error Taxonomy Specific retryable vs. terminal HTTP status mappings Collapsed into generic catch (err) retries; triggers double payouts on ambiguous timeouts.

Asymmetric Payoff in Legacy Production

+ Payoff − Target: Invariants Intact Unbounded downside (dropped invariant or lock = settlement / billing loss) Capped upside (refactoring working paths yields bounded gains)

Batch Size of Risk: Rewrite vs. In-Place

Full Rewrite All invariants risked in one cutover In-Place Steps 1 bounded step verified at a time Shaded area = unverified risk in flight

Figure 3: Legacy revenue systems exhibit asymmetric payoff functions (left) and penalize large cutover batches (right). Upside from cleaner internal syntax is bounded, while dropping a single production invariant creates severe downside loss.

The economic decision rule follows directly from this asymmetry: a rewrite is only net-positive when the discounted maintenance savings of the new code exceed transition engineering cost plus expected tail-risk loss during cutover. When a module replacement is genuinely warranted, it requires five engineering gates before production cutover:

  1. Explicit invariant inventory: A catalog of every state transition, retry rule, timeout bound, and ledger constraint in the legacy path.
  2. Characterization test suite: Replayable tests captured from production request/response payloads and edge-case failure traces.
  3. Shadow or parallel execution: Running the candidate implementation alongside the legacy path on live traffic without mutating state.
  4. Automated reconciliation tolerance: Verifying zero unexplained drift between legacy and candidate outputs over a full business cycle (including month-end close).
  5. Fast traffic rollback and compensating recovery: Feature-flagged routing that can revert incoming traffic to the legacy path in seconds, paired with backward-compatible schemas and compensating journal routines for state already committed.

When those five gates are too expensive to fund, changing the code in place, one verified step at a time, beats a clean-slate rewrite.

Brooks’s Law for Multi-Agent Swarms

Fred Brooks showed in The Mythical Man-Month that adding human developers to a late software project slows it down because coordination overhead scales quadratically (n(n−1)2\frac{n(n - 1)}{2}) with the number of communicators.

Running parallel unconstrained agents (“swarms”) against a legacy monolith reproduces Brooks’s Law at machine speed. If Agent A is modifying settleBatch to support partner fee withholding while Agent B is refactoring account balance queries in the same shared database module, neither agent holds the other’s in-flight semantic assumptions. Even when Git merges their diffs cleanly without line collisions, their combined changes create semantic merge conflicts, such as Agent B introducing a read-lock query path that deadlocks against Agent A’s new fee-account write lock. Senior engineers end up spending more time untangling multi-agent coordination wreckage than it would have taken to write the change by hand.

4. Context Rot, Finite Attention, and Structural Limits

Why does an unconstrained agent loop rack up ~$1,095 in illustrative downstream costs instead of ~$160 per bounded step (~$510 across a three-step feature), producing sprawling, invariant-breaking diffs in the first place? Because iterative context erosion inside the model runtime directly inflates CrediscoveryC_{\text{rediscovery}}, CreworkC_{\text{rework}}, CreviewC_{\text{review}}, and (p×L)(p \times L) in our Section 1 unit-economics equation.

Even when a frontier model advertises a 200k-to-1M token context window, its effective attention budget over complex code invariants degrades rapidly as the session fills with grep output, stack traces, and aborted edits. Researchers call this phenomenon context rot.

First coined in Chroma’s technical report Context Rot: How Increasing Input Tokens Impacts LLM Performance and subsequently corroborated across frontier model generations by Anthropic’s Effective Context Engineering for AI Agents, Adobe Research’s NoLiMa benchmark, Microsoft and Salesforce’s LLMs Get Lost In Multi-Turn Conversation, and iterative coding benchmarks such as SlopCodeBench, long-context degradation stems from three structural properties:

  • The lexical Needle-in-a-Haystack (NIAH) illusion: Standard NIAH benchmarks show near-flat accuracy across 1M tokens only because they test trivial, direct lexical matching. In both Chroma’s 18-model evaluation and the NoLiMa benchmark (which removes literal keyword overlap between query and target), accuracy decays steeply with length, with most models dropping below 50% of their short-context baseline by 32k tokens when latent semantic inference is required.
  • Distractor amplification and coherent haystacks: Legacy repositories are full of topically related distractors: deprecated helpers, similar function signatures, and reverted migration attempts. Chroma’s experiments showed that adding just four distractors sharply accelerates degradation across length, and that models counterintuitively perform worse when the surrounding haystack is logically coherent (such as continuous code and sequential logs) than when sentences are randomly shuffled, because coherent surrounding patterns pull attention away from the target constraint.
  • Focused vs. full context (LongMemEval) and iterative workspace erosion (SlopCodeBench): On LongMemEval, Chroma found that feeding models only the focused relevant context (~300 tokens) consistently outperforms passing the full ~113k-token history, though the magnitude of the drop depends on the model family and thinking configuration (for example, 96% focused vs. 90% full, a 6-percentage-point drop, on Gemini 2.5 Pro with thinking enabled, compared with 92% vs. 38%, a 54-percentage-point drop, on Claude Opus 4 non-thinking and 89% vs. 38%, a 51-percentage-point drop, on Claude Sonnet 4 non-thinking). Crucially, even resetting the chat transcript between tasks does not prevent degradation if the codebase itself erodes: in SlopCodeBench (arXiv:2603.24755), each checkpoint runs in a fresh container with no prior conversation history, yet agents still exhibit progressive structural erosion, cyclomatic bloat, and duplicate helpers as they iterate on their own prior workspace commits.

Why does a larger window hurt reliability even as models improve? A language model does not query its context window the way a database looks up an indexed row. As Anthropic’s Applied AI team notes, transformer self-attention computes n2n^2 pairwise relationships across nn tokens. Every new token stretches a finite attention budget thinner across the sequence.

Continuing with our settleBatch ledger example and building on Drew Breunig’s taxonomy in How Long Contexts Fail and How to Fix Your Context, here are the four specific mechanisms that inflate those downstream cost variables:

  1. Context Poisoning →\rightarrow Inflates CreworkC_{\text{rework}} and (p×L)(p \times L)
    • Mechanism: Early in a multi-turn session, the agent misreads an upstream webhook handler and writes a false assumption into its working scratchpad or summary: “Deadlock retry and account lock ordering are already enforced upstream by the webhook consumer.” Once that unverified claim enters the context window, the model treats its own note as ground truth and strips the lexicographical account_id .sort() out of settleBatch to “simplify” the function.
    • Economic Impact: Every subsequent turn builds on a poisoned premise: stripping .sort() from settleBatch causes either a 60-minute human rewrite (Crework=$150C_{\text{rework}} = \$150) or a production deadlock (p×L=$400p \times L = \$400 expected loss).
    • Architectural Fix: Never persist architectural claims into durable agent memory or scratchpads unless they are backed by a deterministic check or validated checkpoint.
  2. Context Distraction →\rightarrow Inflates CrediscoveryC_{\text{rediscovery}} and CreviewC_{\text{review}}
    • Mechanism: As the agent searches the repository to understand how SettlementEntry objects are constructed, it dumps dozens of tangential files into the session history. Once a trajectory grows past roughly 100k tokens of past actions and log outputs, that history overpowers training-time reasoning. When the harness triggers automatic context compaction to stay under token limits, the model summarizes its own summaries, producing a “photocopy of a photocopy” where general patterns survive but exact constraints (like strict integer minor-unit rounding or lexicographical lock sorting) vanish.
    • Economic Impact: Lossy summaries force repeated repository greps (Crediscovery≈$25C_{\text{rediscovery}} \approx \$25) and produce imprecise, sprawling diffs that require a 90-minute senior audit (Creview=$225C_{\text{review}} = \$225).
    • Architectural Fix: Replace lossy transcript compaction with externalized state on disk and pre-indexed call-graph slices loaded on demand (as used in Anthropic’s tool-result clearing and file-based memory and our harness).
  3. Context Confusion →\rightarrow Inflates CgenC_{\text{gen}} and CreviewC_{\text{review}}
    • Mechanism: Teams often connect coding agents to every available MCP server and local CLI tool at once, exposing 40+ tool schemas (issue trackers, cloud deployers, database shells, doc searchers, multiple file editors) in the system prompt. In the Less is More tool-use study (arXiv:2411.15399), a model failed the GeoEngine benchmark when offered all 46 tools, yet passed the exact same task when offered only 19 tools; similarly, RAG-MCP (arXiv:2505.03275) found that tool descriptions begin overlapping above 30 tools and that curating the active loadout yields up to 3× higher tool-selection accuracy.
    • Economic Impact: Bloated tool schemas burn extra inference turns (Cgen≈$15C_{\text{gen}} \approx \$15) and create noisy side-effect edits across unrelated configuration files that slow down human verification (CreviewC_{\text{review}}).
    • Architectural Fix: Scope the tool surface strictly to the current execution phase: read-only graph retrieval during planning, and minimal file-mutation and local-test tools during bounded implementation.
  4. Context Clash →\rightarrow Inflates CrediscoveryC_{\text{rediscovery}} and CreworkC_{\text{rework}}
    • Mechanism: Suppose the agent’s first two attempts to add partner fee withholding to settleBatch fail the local test suite. In a single long-running chat loop, those two failed diffs, their compiler errors, and the model’s wrong hypotheses remain in the transcript, competing for attention with the third attempt. As Microsoft and Salesforce researchers showed in LLMs Get Lost In Multi-Turn Conversation (arXiv:2505.06120), when early assumptions and later corrections coexist in history, accuracy drops by an average of 39% across frontier models, and the agent begins oscillating between its previous broken approaches.
    • Economic Impact: The loop pays repeatedly in multi-turn tokens to re-read its own failed diffs (CrediscoveryC_{\text{rediscovery}}) and produces chimeric code that blends fragments of aborted approaches and has to be untangled by hand (CreworkC_{\text{rework}}).
    • Architectural Fix: Enforce structured context window rollover, discarding polluted attempt transcripts and starting each bounded step in a clean context window seeded only with the verified plan and current git state.

Context Rot & Context Engineering in Long-Running Agent Loops

Context Rot: Accuracy vs. Input Tokens (Schematic)

High Mid Low 1k 16k 64k 256k 1M Input Context Length (Log Scale) Lexical NIAH (exact match) Semantic needle (NoLiMa / 0 dist.) +4 distractors & coherent logs

Curated vs. Overloaded Context (Schematic)

High Mid Low LongMem NoLiMa Multi-Turn Tool Sprawl Curated / focused Bloated / accumulated
Illustrative Anatomy of a 1-Hour Legacy Migration Window (Finite Attention Budget) Every token competes for n² pairwise attention
Prompt (5%) AGENTS.md + 46 Tools (11%) Action History (28%) Raw Tool Outputs & Logs (53%) Poisoned Self-Note (3%)
1. Context Poisoning →\rightarrow CreworkC_{\text{rework}}, (p×L)(p \times L) Fix: Validate
False scratchpad note strips .sort() from settleBatch, causing a 60-min human rewrite ($150) or production deadlock ($400 expected loss). Validate claims before persisting notes.
2. Context Distraction →\rightarrow CrediscoveryC_{\text{rediscovery}}, CreviewC_{\text{review}} Fix: Offload to Files
Lossy auto-compaction ('photocopy of a photocopy') forces repeated repo greps ($25) and sprawling diffs requiring a 90-min senior audit ($225). Store state on disk and load graph slices on demand.
3. Context Confusion →\rightarrow CgenC_{\text{gen}}, CreviewC_{\text{review}} Fix: Curate (19 vs. 46 tools)
Bloated tool schemas (46 vs. 19 tools) burn extra inference turns ($15) and create noisy side-effect edits that slow down human verification. Scope tools to the active execution phase.
4. Context Clash →\rightarrow CrediscoveryC_{\text{rediscovery}}, CreworkC_{\text{rework}} Fix: Prune / Fresh Window
Failed settleBatch attempt diffs remain in history (-39% multi-turn drop), paying repeatedly to re-read failed diffs and producing chimeric code. Roll over to a clean window per step.

Figure 4: Schematic summary of long-context degradation patterns inspired by Context Rot, NoLiMa, Multi-Turn Sharding, and Tool Loadout studies (top; curve shapes and bar heights are illustrative rather than pooled benchmark measurements), an illustrative anatomy of a 1-hour migration window competing for n² pairwise attention (middle), and the four subtractive context-engineering fixes mapped to downstream cost variables (bottom).

The Effect of Unconstrained Agent Loops

Unconstrained Agent Loops Create… Raw model + long context + no deterministic gate
  1. Lossy Context Amnesia Summaries discard verbatim paths, constraints, and test evidence
  2. Guardrail Gaming Pass-oriented edits weaken assertions or bypass checks
  3. Silent Cross-Service Coupling Unbounded edits modify shared helpers across service boundaries
  4. Rediscovery Token Tax Repeated repo searches and uncached context rebuilds
  5. Higher Defect Injection Dropped constraints reintroduce previously ruled-out regressions
  6. Review Queue Stalls Large, unverified diffs accumulate in senior review

Figure 5: Modeled on Reinertsen’s causal analysis of unmanaged queues, unconstrained agent loops generate six compounding liabilities across verification cost, defect rates, and review throughput.

5. Governance Controls and Economic Trade-Offs

You cannot solve context rot or invariant erosion by pleading with the model in a system prompt (“Please be careful and do not edit tests”). Prompts are probabilistic suggestions; legacy systems require deterministic guarantees. Worse, trying to patch behavioral drift by stacking all-caps warnings into system instructions creates what Breunig calls prompt debt: brittle natural-language rules that regress other instructions and lock the workflow to a single model’s quirks.

To bring the illustrative cost from ~$1,095 for an unconstrained loop down to ~$160 per bounded step (~$510 across a three-step feature), the model must be wrapped in a bounded execution harness. When we run the settleBatch partner-fee change inside a disciplined harness (such as mux and muxcode), five structural controls alter the economics of the run:

  1. Pre-Execution Plan & Path Contracts: Before any mutating tool is unlocked, a planning step scopes the change to settlement/settleBatch.ts and settlement/postings.ts. Attempts to touch unrelated upstream webhook handlers are rejected by the host gate before a single line is written.
  2. Immutable Test and Contract Locks: During the implementation step, settlement/settleBatch.spec.ts, database migration history, and OpenAPI/Protobuf schemas are protected by the host gate as read-only. When the agent’s naive fee deduction fails UnbalancedLedgerError or expect(receipt.totalDebits).toBe(receipt.totalCredits), the host blocks any attempt to edit settleBatch.spec.ts and runs a property-based zero-sum invariant gate. Forced to satisfy the invariant, the agent splits the primary gross settlement entry into a net merchant credit (primary.amount - feeAmount) and an offsetting partner-fee credit (feeAmount) so the ledger foots to zero.
  3. Pre-Indexed Dependency Graph Context: Instead of burning $20 in tokens grepping the repository, mux queries a pre-indexed code graph and injects a compact, 6,000-byte commit-bound context packet containing settleBatch and its two-hop import and call-graph neighbors (lockAccountsForUpdate and buildDoubleEntryPostings), keeping the lexicographical account_id sort invariant immediately visible in high-attention context.
  4. Deterministic Host Gates: AST linters, typecheckers, and the characterization test suite execute locally in muxcode after every candidate edit, filtering out broken runs before a pull request is ever opened, and before an independent judge on a different model reviews the change.
  5. Structured Window Rollover & Confidence-Gated Sizing: Each phase (Plan→Implement→Verify\text{Plan} \rightarrow \text{Implement} \rightarrow \text{Verify}) runs in a fresh process with a clean context window and a deterministic file handoff, clearing prior attempt transcripts to reduce transcript-level Context Clash and Context Distraction. Meanwhile, Clef (a calibrated decision model) sizes the workflow per task, routing well-bounded single-module edits to fast worker models while reserving full multi-model verification for changes where mistakes could slip past tests.

The table below maps each governance control directly to the cost variable it compresses, alongside its operational trade-off and the telemetry metric used to verify it:

Governance Control Primary Cost Reduced Operational Trade-Off Observable Telemetry Metric
1. Pre-Execution Path & Batch Contracts CreviewC_{\text{review}}, CdelayC_{\text{delay}} Requires decomposing cross-cutting features into sequenced small PRs Median files touched per PR (≤3\le 3) & PR time-to-first-review
2. Read-Only Test & Schema Mounts (p×L)(p \times L), CreviewC_{\text{review}} Requires a distinct, human-approved step when tests genuinely need updating Zero unauthorized test/schema mutations in implementation runs
3. Pre-Indexed Call-Graph Context Packets CrediscoveryC_{\text{rediscovery}}, CgenC_{\text{gen}} Indexing compute overhead on git push and periodic index refresh Ratio of search/read tokens to final diff tokens
4. Deterministic Local Host Gates CreworkC_{\text{rework}}, CreviewC_{\text{review}} Slower local step completion when gates reject bad diffs First-time CI pass rate & post-PR human rework commits
5. Clean Window Rollover & Sizing CrediscoveryC_{\text{rediscovery}}, CreworkC_{\text{rework}} Requires structured checkpoint handoff between steps; misrouted runs re-enter Token cost per completed step & retry oscillation rate

(For a deep dive into the mechanical implementation of these controls, including how muxcode enforces permission cards in its local Rust host, spawns isolated pi RPC processes per task for clean window rollover, and builds commit-bound context packets via mux, see the companion post, Durable Harness for Coding Agents.)

When a harness is uneconomical: Formal contracts, host gates, and multi-stage review are not free. For greenfield prototypes, disposable scripts, or isolated UI experiments with zero shared state and trivial rollback cost, the upfront latency of contract generation and strict path gating exceeds the expected downside risk. Governance pays for itself only where state is persistent, coupling is non-trivial, and senior review capacity is the binding constraint.

6. Rollout Playbook: Adopting Bounded Agents in Legacy Repos

Transitioning an engineering organization from ad-hoc, unconstrained agent chatting to bounded harness execution does not require freezing feature delivery. It requires sequencing adoption around your highest-risk boundaries in four phases:

Phase 1: Instrument Baseline Unit Economics (Weeks 1 to 2)

Before changing workflows, stop reporting vanity metrics like “lines of AI code written” and establish a baseline for Total Cost per Accepted Change:

  • Track PR cycle time split by diff size (specifically measuring the queue delay jump when diffs cross 5+ files).
  • Track post-opening rework commits and CI failure loops on agent-assisted PRs.
  • Audit recent production incidents or near-misses to identify the top three modules where undocumented invariants live (e.g., ledger postings, idempotency claims, settlement batching).

Phase 2: Lock the Boundaries (Weeks 3 to 4)

Before allowing agents to iterate autonomously inside legacy critical paths, harden the verification perimeter:

  • Write characterization tests around high-risk invariants (such as double-entry balance footing and lexicographical lock acquisition in settleBatch).
  • Configure your agent harness to enforce read-only guards on *.spec.*, *.test.*, schema definitions, deployment manifests, and .git hooks during code-writing turns.

Phase 3: Enforce Plan-First, Small-Batch Execution (Weeks 5 to 8)

Shift developers from open-ended repo chat loops to plan-gated execution:

  • Require every non-trivial task to produce a bounded file-path manifest before write tools are enabled.
  • Cap implementation steps to diffs that a senior engineer can verify in a single 15-to-20-minute calendar window, and enforce hard WIP limits per reviewer. If a feature requires touching eight files across three services, decompose it into three contract-locked changes.

Phase 4: Pre-Index Context and Route by Confidence (Ongoing)

Compress the rediscovery token tax and scale throughput safely:

  • Replace repetitive repo-wide grep loops with pre-indexed dependency and call-graph retrieval so every agent step starts with the exact two-hop neighborhood of the target symbol.
  • Use confidence-gated routing to send well-bounded, single-module edits to fast, cost-effective worker models while reserving frontier reasoning models for architectural planning and invariant verification. Expand to adjacent services only if total cycle time and loaded cost per accepted change decrease while change-failure rate and reviewer queue age remain flat or improve.

7. Conclusion

In legacy software engineering, the bottleneck has never been the speed at which developers can type characters into an editor; it is the speed at which the organization can safely verify that a change preserves the system’s invariants.

When we treat coding agents as magical authors and optimize for cheap, unconstrained token generation, we flood our most constrained bottleneck, senior engineering review, with sprawling, complected diffs. Review queues stall, characterization tests get quietly weakened, and production tail risk spikes.

When we instead treat coding agents as a stochastic subsystem with quantifiable downstream externalities, the engineering strategy becomes clear. By wrapping models in deterministic harnesses that enforce pre-execution scope contracts, read-only test locks, pre-indexed call-graph context, and clean window rollovers, we compress rediscovery, rework, review latency, and incident risk simultaneously. The organizations that win with AI in legacy systems will not be the ones generating the most lines of code per minute; they will be the ones shipping the lowest Total Cost per Accepted Change.

Documentation for the contracts, gates, and shared index described here is at muxco.de and usemux.com.

References: