A method for verifying what AI agents write

The work

New to this? Save yourself some pain and start with the field report. It shows the loop repairing to the data, not the screen — the part I have not found anywhere else — and catching seven faults that every other check passed. Then the paper specifies the whole method, and lands rather better once you understand the mechanics.

In plain terms

An AI agent can write more working code in an afternoon than a team can carefully check in a week. Generation got dramatically faster; checking did not, so the work piles up in front of the review instead of behind it. What you can delegate is set by what you can verify, not by what the agent can produce.

The usual answers help, and none of them finish the job. Tests the agent writes inherit the agent’s understanding, including the part that was wrong; they are a second expression of one opinion, not a second opinion. Code review is strong on what was written badly and weak on what was never written at all: the field nobody carried through, the record nobody wrote. Its cost also tracks the volume under review, which is the quantity that just exploded. And trust is not a verifier: METR has documented frontier models editing tests, obtaining reference answers, and exploiting loopholes in scoring when doing so improved the score.

The idea. For a great deal of business software you already hold something nobody thinks of as a testing asset: a trusted body of real data, in the shape the system is supposed to produce. Call it the gold state, and use it twice. Use it as the question, taking only what a person could actually type in: the names, the amounts, the selections on the screen, not the record’s internal identifiers or its audit trail. Then use it as the answer key: the complete record the system should end up with, including the derived values, the audit rows, the queued notification, the linked children. Drive the real application with the question, and compare what it produced against the answer key.

Isn’t that circular? It would be if the answer key came from the application. It does not. The gold state exists independently, is versioned and protected, and the application under test can neither read it nor write to it. The test store starts from a declared baseline holding none of the scenario’s business data, receives only the input side, and has to construct the result on its own. There is a second answer underneath that one: within a run the gold state is stipulated, because the question being asked is whether the application reproduced it. Whether the gold state is what the business actually wants is a separate question with a separate answer, and somebody has to approve it. The method keeps those apart deliberately, and never asks a run to certify its own reference.

What matching means. Not that the row counts line up. Every in-scope table, record, field, and durable side effect must be accounted for: compared exactly, translated where internal identifiers legitimately differ, checked by a stated rule such as a header total equaling the sum of its lines, or excluded for a written reason with another gate that covers it. The decisive part is what happens to anything fitting none of those categories: it fails. Add a column nobody classified and the comparison goes red until a human decides what that column is. Unknown state is neither ignored nor assumed innocent, and that single rule is what makes the check hard to quietly weaken.

Two paths, because the browser is slow. You would like to push all your data through the real interface, and you cannot: a multimillion-record corpus driven through a real browser can take hours or days. So a small, deliberately chosen set of scenarios goes through the actual screens, with no shortcuts and no test-only back doors. That shows the interface does what it appears to do across the behavior it was built to cover, and it records the exact shape of the instructions the screens send inward. Everything else then goes through the application’s own front door using only those recorded shapes: a case matching nothing the interface demonstrated fails rather than being guessed at. One path shows the screens are honest about what they were asked to do; the other shows the machinery underneath holds across the whole corpus in scope.

What a failure looks like. Not “something broke.” This scenario, this record, this business key, this field, expected this, got that, at this step. That is a fact rather than an impression: small enough to act on, precise enough that a person, an assisted diagnosis, or an agent working unattended can all consume the same artifact. It is also what makes handing the repair to an agent a governed act rather than an act of faith. The oracle, its scenarios, the gold state, and the recorded command evidence are identity-hashed and sit outside the agent’s write authority, and every rerun re-checks the starting state from scratch. The moves that would let an agent optimize against the gate are closed, not merely discouraged: within the limits the paper sets out, fixing the code is the only repair the loop can make on its own authority, and every other surface charges a toll — changing the oracle requires independent approval, and no repair of any kind counts until the deterministic driver re-witnesses it. Not one open door; every other door alarmed. The gate itself is ordinary deterministic software, and removing every agent from the picture changes nothing about how it behaves.

One example, in full — from a fault-injection exercise, where the operator plants defects on purpose and the machinery is not told. A write path persisted the wrong one of two related identifiers into a stored record. The value was real, correctly typed, and wrong — a genuine identifier, of a genuine thing, in the wrong column. Every read agreed the write had succeeded: the record loaded, the screen rendered it, and a test reading the value back would have seen a plausible identifier and passed. It surfaced only when the application resumed from its own stored state and re-submitted — and the corrupted value, re-consumed by the write path that produced it, matched nothing it was required to match. Here is the cycle as the runner recorded it:

RESULT: FAIL — expected a second version; the version strip still shows v1

  Iteration 2/5 | Type: API contract
  Hypothesis: the assembler persists the outer selector’s identifier into
              the inner selector’s field. The first submission succeeds and
              stores the corrupted value; the console resumes from the stored
              record, so the second submission carries an inner selector equal
              to the outer one and matches no template baseline.
  Evidence:   the second submission was refused outright; its request body
              carries the outer identifier in the inner field; the earlier
              options call on the same run shows the true inner value.

-- fix applied; driver relaunched at the failed scenario --
RESULT: PASS (all 35 of the scenario’s tests; fix witnessed)

One column, one identifier, one token. Nothing that merely read the value back could have found it — the shape of corruption that surfaces years later in an export, an integration, or a migration. It was the second of two faults stacked in the same scenario: invisible until the first was repaired, then convicted the moment the stored state was re-consumed. The diagnosis, the repair and the rerun happened with no human in the loop; the verdict that let the chain advance was issued by the deterministic driver, not by the agent that wrote the fix.

What it does not do. It covers only what was modeled and actually exercised; it does not discover behavior nobody listed, and deciding what to cover stays real human work. Reproducing the gold state is not approval of the gold state; somebody still owns whether the reference is right. Volume does not substitute for the interface: a defect living only in a screen path that was never exercised is not addressed by pushing the whole corpus through the front door. And it is one gate among several, silent on security, privacy, permissions, performance, resilience, and whether an email was actually delivered. It does not replace code review either; it replaces the grinding part where a human reconstructs from a diff what every stored value was supposed to be. Judgment about architecture, risk, and the reference data itself stays exactly where it was.

This section is an introduction and is stated loosely on purpose. The claims that matter, and the limits that bound them, are in the paper.

The method, stated precisely

The central mechanism is gold-state parity™. A trusted, fixed, versioned gold state supplies both the scenario inputs and the expected persisted result. The test store is never seeded with gold: it begins from a declared baseline, receives inputs only through a declared application entry path, and must reproduce gold’s expected-state projection under an explicit, fail-closed parity contract. One corpus supplies stimulus and expectation without circularity.

The standing gate has two lanes. Proof drives a coverage-selected subset through the real user interface, mechanically evaluates the declared interface obligations, establishes browser-to-persistence state parity, and captures the normalized command-contract variants the UI emits. Validate instantiates only those captured variants with the full in-scope corpus through the application’s official command entry point, tethered to the current Proof evidence by identity and an out-of-band provenance witness.

A failing gate is engineered to be as consumable as a passing one. A covered disagreement surfaces as a mechanical, localized artifact naming tables, business keys, and field-level mismatches, built to feed a diagnose-repair-rerun loop, whether agent-operated or human-operated, rather than to be reconstructed from a diff. The gate itself stays deterministic, with no agent judgment in its accept/reject path.

A passing gate establishes contract-exact observational equivalence at the declared persistence boundary for the identified build, runtime profile, and oracle version, and nothing outside the versioned coverage model, which is the principal residual limitation and matures across immutable oracle versions. Read-path rendering, security, performance, concurrency, and external delivery remain separate gates.

Try it on one write path

The gate the paper specifies is a standing gate over a whole corpus. To see the mechanism work before committing to that, run it on one write path.

Pick a single create or edit path that has an official entry point — a real screen, or an API the product actually publishes. Freeze a small gold fixture for it: the values a scenario enters, and the exact rows that must exist afterward, held outside the write authority of whatever implements the path. Declare the starting state, and verify it before the run, so nothing can pass by inheriting an answer it never produced. Enter the values through that official door, choosing values that cannot be confused with a default, an empty value, or a dropped field. Compare the full closure of what was written, not the fields you thought to check, and fail closed on anything unclassified. Then make the failure name the entity, the business key, the field, the expected value, and the actual value.

That is a proving slice, not a conforming gate: no coverage model, no captured command contract, no second lane, and no independent governance of the fixture, so it supports none of the paper’s pass claim. What it tests, on one path, is whether what the application actually persisted matches the state the fixture declared it must produce — which is the question the whole method is built around.

Preferred citation

Fahnestock, M. (2026). Expectation-Driven Development: Gold-State Parity™ as a Write-Path Acceptance Gate for Agent-Written Stateful Applications (Version 1.0). https://doi.org/10.5281/zenodo.21761800. Licensed under CC BY 4.0.

@misc{fahnestock2026edd,
  author  = {Fahnestock, Melvin},
  title   = {Expectation-Driven Development: Gold-State Parity™ as a
             Write-Path Acceptance Gate for Agent-Written Stateful
             Applications},
  year    = {2026},
  version = {1.0},
  doi     = {10.5281/zenodo.21761800},
  url     = {https://doi.org/10.5281/zenodo.21761800},
  note    = {Licensed under CC BY 4.0}
}

This work is self-published and has not been peer reviewed.

Trademark

Gold-State Parity™ is a trademark of Melvin Fahnestock. It identifies the mechanism defined in this work, and the tools and services built on it.

The reservation is about the name, not the technique. Section 2(b) of CC BY 4.0 expressly excludes trademark rights from the license, and they are reserved. That reservation does nothing to restrict implementing the method, describing it, teaching it, or writing about it — including by name, in the ordinary descriptive and referential ways. No permission is required, and none of this is a claim over the technique itself, which stays free to implement.

What the mark distinguishes is source: the author’s own publications, tools, and services. The one thing it asks is that you not use it in a way suggesting your product or offering originates with, or is endorsed by, the author.