EXECUTION SURFACES AND VERIFICATION
5. Execution Surfaces and the Enterprise Cost Model
Where a loop physically executes determines two things an enterprise platform team must own: whether a machine needs to stay awake, and which billing or resource-allocation contract governs the run. Four surfaces span an escalating autonomy gradient:
| Surface | Characteristics | Machine state | Typical billing route |
|---|---|---|---|
| In-session | Interactive, prototyping | Awake, session open | Subscription / allocated seat |
| Desktop scheduled task | Independent session per run, fresh context each time | Must remain awake (with a catch-up window on wake) | Subscription |
| Cloud routine | Runs on managed infrastructure, laptop can be off | N/A | Currently subscription, policy in flux |
| Headless / SDK / CI integration | Fully decoupled automation | N/A | Per-token API billing |
The practical guidance for enterprise platform design: prototype in-session, promote proven loops to a scheduled surface for daily use, and only fall through to a fully decoupled headless or SDK path when automation must run independent of any interactive account context (for example, triggered by a CI event on a shared service account). This “promotion path” — session → scheduled task → managed routine → headless — mirrors the maturity curve most enterprises already apply to moving a script from a laptop to a scheduled job to a managed pipeline, and it is worth encoding as policy rather than leaving to individual engineer discretion.
A specific cost-control finding worth flagging to any FinOps or platform-cost function: headless, non-interactive invocation of an agent can silently route to per-token API billing even under an otherwise-unlimited subscription arrangement, if the invocation environment happens to carry API credentials. Enterprises building scheduled or CI-triggered agentic automation should verify the active billing route before wiring anything into a cron-style trigger, and should treat “which contract is this landing on” as a pre-deployment checklist item, not an assumption. One documented incident saw a subscriber accumulate several thousand dollars in unexpected per-token charges over two days by scheduling headless invocations that silently fell onto the API billing path rather than the subscription pool they believed covered it.
Because this policy area moves quickly — billing boundaries between subscription and API-metered usage were adjusted, paused, and are expected to shift again — any cost model built into an internal platform should be re-verified against current vendor documentation before being treated as load-bearing, and should never be hard-coded into automation without a periodic revalidation step.
6. Verification: The Central Architectural Decision
If a single decision determines whether an enterprise’s agentic loops are trustworthy, it is this one: the agent that produces work and the agent that judges it must be separate.
6.1 Why self-grading fails
A single model instance evaluating its own output exhibits the same bias a human author exhibits reviewing their own draft: it will find what it expects to find, and it will not surface its own blind spots. This is not a capability gap that improves with a better model — it is a structural property of self-assessment. The fix is architectural, not a matter of prompting the model to “be more critical.”
6.2 What real verification requires
A verification step is only meaningful if it could plausibly fail even when the producing agent is confident the work is correct. Re-running the same unit tests the producing agent wrote inherits that agent’s blind spots — if the tests encode a misunderstanding, so does their re-execution. Stronger verification strategies, roughly ordered from weakest to strongest:
- Independent test execution with raw output — not a summary, the actual output, pasted into the transcript.
- User-level exercise of the feature — starting the service and driving the actual path a change claims to fix, rather than trusting internal assertions.
- Golden or snapshot comparison against a known-good fixture.
- Property-based or fuzz testing — invariants checked against generated inputs rather than hand-picked cases that the author may have unconsciously chosen to pass.
- Schema or contract validation against a published interface definition.
- Visual regression for UI-facing changes.
- Independent re-derivation — a second agent solving the problem by a different method, with results compared. This is particularly valuable for financial or numerical logic, where a test suite written by the same author who wrote the calculation will often encode the same misunderstanding as the calculation itself.
- Lint, type, and build checks — necessary as a floor, but never sufficient as proof of correctness on their own.
An enterprise checker agent’s system prompt should explicitly instruct it to trust nothing it cannot reproduce itself, and to report a failing verdict with specific reproduced evidence rather than accepting the producing agent’s description of its own work. The strongest possible design gates the loop’s completion condition on the checker’s independently reproduced pass — not on the producing agent’s self-report — so the loop structurally cannot terminate on a self-assessment.
6.3 Verifying the right thing
The subtle and recurring failure is a checker that verifies a proxy for the goal rather than the goal itself: tests that pass without covering the actual defect, a green build that says nothing about runtime behavior, a lint pass mistaken for correctness. The diagnostic question worth institutionalizing in code review and platform design alike: if this check passes and the feature is still broken, what did the check fail to catch — and then add the check that would have caught it.