Verifiable Finish Lines: an Agent Stopping Is Not Proof the Job Is Done
A practical completion contract for AI-agent work: define the artifact, checks, evidence, bounds, and terminal states before the run starts.
An agent that stops is not necessarily an agent that finished. A run can end because it hit a token cap, timed out, paused for approval, lost a dependency, or decided its own work looked fine. Those are control-flow facts. They are not proof that the outcome exists, that it is acceptable, or that a side effect really happened.
The practical fix is a verifiable finish line: a completion contract written before the run starts and checked when it ends. This guide calls that practice “verifiable finish-line design.” That is an editorial synthesis, not an established industry term. Its value is simple: it separates the process ended from the job was verified complete.
The discipline behind the idea
The instinct to define done before beginning is older than AI agents. The Scrum Guide defines a Definition of Done as a formal state that meets agreed quality measures. Agent systems add practical controls: structured outputs can constrain an artifact’s shape, while guardrails and human review distinguish automatic checks from approval before sensitive actions.
There is also a useful warning. Google DeepMind defines specification gaming as satisfying the literal objective without achieving the intended outcome. A perfectly formatted report with fabricated sources has passed a shape check and failed the task. A successful publish call may have reached an API without making the approved artifact live. A finish line needs to test the outcome you actually care about, not just the easiest proxy.
The eight-part finish-line card
Write this card before launching an agent. If a row is unknown, that is a reason to stop and clarify the work, not a reason to let the agent invent a definition of success.
1. Outcome
Name the result in observable terms. “Research competitors” is an activity. “Produce a comparison of the ten named competitors with current primary-source evidence for price, capability, and limitation” is an outcome.
2. Artifact or state change
Name the object that must exist: a file at a path, a validated JSON record, a pull request, a database row, or a published URL. For external actions, name the expected state change as well as the request that proposes it.
3. Hard gates
List the binary conditions that cannot be averaged away: required fields, schema validity, mandatory citations, an approved target, a permission boundary, or a read-back of the exact record written. Hard gates are especially useful for safety, authorization, privacy, and irreversible actions.
4. Quality checks
Some questions are not binary. Accuracy, relevance, clarity, and decision usefulness need a rubric, examples of failure, and a named reviewer. A schema tells you whether a required key exists; it does not tell you whether the recommendation under that key is sound.
5. Evidence receipts
Decide what the run must preserve: inputs, source URLs and retrieval times, validator output, artifact hash, run ID, approvals, and the external-state read-back. The National Academies’ definition of reproducibility emphasizes consistent results under the same inputs, steps, methods, code, and conditions. For ordinary agent work, that makes a strong model for a receipt without pretending every run is a scientific study.
6. Bounds
Set limits for time, retries, tool calls, cost, and allowed write targets. A cap is an emergency brake, not a success condition. State what happens when it is reached: preserve partial work, report the missing gate, and route the next decision to an owner.
7. Authority
Say which actions are automatic and which pause for a person. The approval boundary should sit before consequential side effects, not after the agent has already changed the thing that needed review.
8. Terminal states
Use statuses that describe reality: completed_verified, completed_needs_review, failed_validation, budget_exhausted, blocked, paused_for_approval, and tool_error are a useful starting set. “Stopped” alone is not a status. Each ending must say what evidence exists, what did not happen, and who owns the next move.
A research-dossier example
Imagine asking an agent for a research dossier. The finish line could be:
- Outcome: one decision-ready dossier on a named topic.
- Artifact: a markdown file at an agreed path plus a verification report.
- Hard gates: required sections exist; every cited URL resolves; every high-risk claim has a primary or authoritative source; the storage upload is read back.
- Quality checks: a fresh-context reviewer checks source fit, gaps, and whether the summary answers the stated decision.
- Evidence: source inventory, retrieval times, URL results, artifact hash, reviewer report, and storage metadata.
- Bounds: a fixed research window and source-pull cap; exhaustion produces
budget_exhausted, not a padded report. - Authority: publication requires the assigned editor or an explicit automation authorization.
- Stop rule: mark
completed_verifiedonly when the artifact exists, all hard gates pass, the independent review is clear, and the storage read-back matches.
This is more work than “research it deeply.” It is also far easier to diagnose when it fails. If one source is unreachable, the status is failed_validation or blocked; nobody needs to pretend the agent completed a task it cannot support.
Three traps to avoid
Passing the proxy instead of the outcome
A checklist can cover headings, word count, and links while missing whether the work answers the decision. Combine measurable gates with an outcome-focused rubric and a review of known failure cases. Passing recorded checks is useful evidence; it is not proof that every dimension of the intended outcome was captured.
Letting the agent certify itself
Self-review can catch obvious mistakes and improve a draft. It is not independent certification, especially when money, publication, safety, or reputation is involved. Use a fresh-context reviewer, deterministic validators, primary-source retrieval, or a human approval gate according to the risk. Separate evidence paths matter more than merely asking the same agent to review its own answer again.
Calling a cap a success
Timeouts, retry exhaustion, approval pauses, and tool errors are valid terminal events. None mean the task succeeded. Preserve the partial result, report which condition remains unmet, and decide whether to resume, revise the contract, or hand the work to a person.
Done means
A run is done only when the named artifact or state change exists, every hard gate has passed, required evidence is stored, the right reviewer or approval has occurred, and no bound was used as a substitute for a pass. For an external action, read back the exact target after the write. A successful response is weaker evidence than the resulting live record, URL, or object.
What this article does NOT cover
- A universal optimum for deterministic checks, model graders, repeated trials, and human review.
- Vendor-specific implementation code or a claim that any product feature is permanently available.
- A guarantee that a well-written completion contract makes an agent objectively correct.
- A substitute for legal, safety, compliance, or organizational approval processes.
Related guides
Research basis: Johnny5/Websites/Agentic Botsitting/research/verifiable-finish-lines/verifiable-finish-lines-research-2026-09-05.md (verified 2026-09-05).
Sources
Sources
Submit a take
Have a different read on this? Drop a comment below — your email isn't published, and I read every one. Nothing leaves the site until I approve it.