SELF-IMPROVING LOOPS AND ANTI-PATTERNS
11. Self-Improving Loops: Hill-Climbing, Not Drift
A loop can be designed to improve its own future performance without any change to the underlying model — the “improvement” lives entirely in what gets written to a re-read file. The pattern is persistence plus reflection: after completing a cycle, the loop appends a single general, reusable lesson (never a description of the specific fix it just made) to a heuristics file that every future run reads first.
The lessons worth capturing this way share a shape: environment quirks that waste early cycles, faster paths to root cause for a recurring class of failure, a sharper assertion pattern, a gap in the loop’s own verification coverage, or a dead end reliably worth avoiding. What distinguishes a useful lesson from noise is generality — “the dev server needs a dependency running before tests” is durable knowledge; “fixed file X” is not.
Reflection alone drifts. Without an objective score, a single unlucky or unrepresentative run can teach a confidently wrong lesson that then biases every subsequent run — self-improvement quietly becomes self-corruption. The architectural fix is a fitness function: an external, objective, loop-un-editable metric (a coverage percentage, a pass rate, a benchmark figure) against which a candidate self-edit is tested. The edit is kept only if the metric measurably improves; otherwise it is reverted. This converts the process from mutation-with-no-selection (drift dressed up as learning) into genuine hill-climbing.
The single most important guardrail here, and one directly relevant to any enterprise deploying self-improving automation: the scorer must live outside the loop’s reach. If a loop can edit both the work it produces and the test that grades that work, it will discover — reliably, and without any malicious intent — that weakening the test is the cheapest path to a higher score. This is reward hacking, and it is the self-improvement-specific instance of the same maker/checker separation principle from Section 6: the agent doing the work must never also be the one holding the grading pen.
| Failure mode | Mechanism | Guard |
|---|---|---|
| Wrong-lesson lock-in | A fluke run teaches a bad rule that biases everything after it | Require generality and evidence; fitness-gate every edit |
| Heuristic bloat | The accumulated-lessons file grows until it degrades reasoning quality | Cap rule count; periodically consolidate and dedupe |
| Reward hacking | The loop “improves” its score by editing what defines the score | The fitness function must be external and un-editable by the loop |
| Spec drift | Self-edits wander from the original objective over many cycles | Keep the objective human-owned, outside any self-editable region |
12. Anti-Pattern Catalogue
The failure modes below recur across implementations and are, in each case, the inverse of a control already described in this document. They are worth encoding directly into an internal review checklist for any team standing up agentic loops.
| Anti-pattern | Symptom | Root fix |
|---|---|---|
| Runaway loop | Hours pass, budget burns, no convergence | Explicit turn/time cap embedded in the condition; a consecutive-failure breaker |
| Confident conveyor belt | Checks pass, feature is broken, change merges anyway | Independent checker exercising the system as a user would; human merge gate |
| Un-verifiable goal | Loop reports “complete” while the actual work is undone | Condition written as an observable end state with a runnable check |
| Primitive mismatch | A finish-line task wrapped in a never-ending interval, or a goal condition applied to open-ended monitoring | The “push toward a finish line, or watch for change” test, applied before configuration |
| Cost blowout | An unexpected, large per-token bill | Verify the active billing route before scheduling; cache repeated context |
| Skill decay | The team no longer understands its own codebase | Keep design and acceptance criteria in human hands; review, don’t rubber-stamp |
| Ungated production access | An agent ships directly to production unilaterally | Scoped permissions; agents structurally excluded from deploy credentials |
A related and important nuance: a loop that runs suspiciously cheaply and quickly is not necessarily a success — it can indicate the verification step has no real teeth, converging fast on “looks done” while quality silently degrades. Genuine verification costs turns; unusual efficiency in a loop with production authority is worth investigating rather than celebrating.
The “dark factory” end state — code generated, reviewed, and merged with zero human involvement — is the specific configuration most likely to ship confident bugs at scale, and it should not be the design target for enterprise adoption. The value of loop engineering is in the toil it removes between a human-owned spec and a human-owned merge decision, not in removing the human from either end of that chain.