Skip to content
Cyborg Mathematics
PaperAccepted

Verify Before You Accept

A case study of my own AI agent team: what the lead agent's check before accepting work caught, and what got past it.

Accepted as a poster at the Lifelong Agents workshop of the AIAS+ symposium in San Francisco, and free to read on Zenodo. The paper reads the records of my own research program to see how the lead agent's check before accepting work, and the rules written after each incident, held up.

October 20, 2026· updated Nov 7, 2026

Status

"Verify Before You Accept: 33 Days of Incident-to-Rule Practice in an Operator-Supervised Agent Hierarchy" was accepted on as a poster at the 3rd Workshop on Lifelong Agents: Learning, Aligning, and Evolving, part of AIAS+ 2026, and I presented it as a poster at the workshop in San Francisco on November 7, 2026. The workshop is non-archival, so the paper is not part of a formal proceedings. The final version is free to read on Zenodo (doi:), together with the de-identified tables and the scripts that built its counts.

What the paper reports

My research program runs as a small hierarchy: I set the scope and decide anything that cannot be undone or that will be seen outside, a lead Claude agent plans the work and accepts or rejects what other agents report, and those agents compute, check and draft. The lead agent accepts a result only after re-running or checking it, and every incident becomes a written rule for later agents to follow. The paper reads days of the program's own records to see what those two practices did. It counts runs: a run is one task a helper agent carried out and reported back to the lead agent, such as computing a number or checking a claim.

At acceptance, the check changed of the runs the lead agent ruled on, and only one of those was for a wrong number. The records show at least ▲ of the accepted runs were found wrong later. The paper uses this to separate two things a check can do: reproduce a result, and establish that it is correct.

The file of rules twice recorded one of my instructions with a restriction I had not given, and the lead agent acted on the wrong version before I corrected it. Of the recorded incidents, two coders judged to be repeats of a problem a rule in force already covered and unclear. The coders were two Claude models, not people, working blind to each other and to the paper under a rubric fixed before the coding began.

The record comes from one program, with one operator and one vendor's models, and the paper does not estimate what the checks prevented or missed.

Claude models (Anthropic) assisted this work under my direction. They wrote and ran the scripts that build the paper's tables, re-checked its references and numbers against the records, and assisted with the writing throughout. As the paper's own disclosure states, the text is mine and I take full responsibility for it.

Notes on the numbers marked ▲

  • Accepted runs later found to have delivered or kept something wrong: 11. At least this many. The records hold only the errors someone noticed.

These notes appear in every version of this page.

Revision history

  1. 2026-11-07 · status-change

    Presented as a poster at the workshop in San Francisco.

  2. 2026-11-06 · created

    Added on the day the paper went up on Zenodo, so the site lists all of my accepted papers.

How to cite this

Tyler Satchel Orden (2026). Verify Before You Accept [Workshop poster, non-archival]. https://doi.org/10.5281/zenodo.23652450

DOI: 10.5281/zenodo.23652450

BibTeX
@unpublished{orden2026verifybeforeyouaccept,
  author = {Tyler Satchel Orden},
  title  = {Verify Before You Accept},
  year   = {2026},
  note   = {Workshop poster, non-archival. Retrieved 2026-11-07},
  doi    = {10.5281/zenodo.23652450},
  url    = {https://cyborgmathematics.com/papers/verify-before-you-accept}
}

See also