Don't Ship It. Prove It First.

Merging a paragraph, not a proof

"Implemented rate limiting — tested, working." Merged on the sentence. Production taught the lesson the sentence skipped: no test existed, the limiter counted per-process (useless in cluster), and the first scraper proved it within hours.

The transcript had confidence. It lacked the one thing confidence can't replace: output of commands run against reality. Literature had been accepted as engineering.

Don't ship it. Prove it first. Four words that would have saved that week — now taped above every merge this team makes.

Claims vs evidence at merge time

Three things transcripts teach the hard way:

1. Confidence is a style, not a signal. Models write "definitely fixed" and "might be fixed" with equal fluency — tone carries zero information about correctness. Merges keyed to confident language approve fiction at the same rate as fact. Read claims as weather (atmosphere), proof as ground (footing).
2. Untested paths are guilty until proven innocent. The rate limiter failed on the exact path nobody curled (multi-process reality vs single-process tests). Every unexercised path is a suspect; merges are verdicts. No curl, no verdict — the case stays open and the suspect ships free.
3. Re-running beats re-reading. Re-reading that transcript twice convinces both times; re-running its claims once catches it in seconds. Reading evaluates rhetoric; running evaluates reality. Thirty seconds of commands outperforms thirty minutes of careful reading, every single merge, forever.

The done-but-500 post showed the corpse. This one runs the autopsy as policy.

Ai verification gate ritual

Claim, evidence, independent re-run — the three-part merge requirement in the lock file.

Step 1 — Demand the proof bundle with every handoff.

CLAIM: rate limiting enforced per user across cluster.
PROOF:
  gate: [pasted: node --check + lint, exit 0]
  happy: [pasted: curl 200 with normal traffic]
  unhappy: [pasted: curl 429 past limit from two workers]
  adversary: [pasted: spoofed user id → still limited]

Four pastes, no prose accepted as substitute. The template converts "tested, working" from literature into exhibits — numbered, timestamped, re-runnable. Agents adapt within sessions (proof becomes reflex when merges depend on it); transcripts get shorter and truer simultaneously.

Step 2 — Re-run the bundle yourself (or let CI).

# your terminal, their branch, two minutes:
npm run lint && curl <their happy> && curl <their unhappy> && curl /api/health
# match? merge. mismatch? the transcript lied — reject regardless of prose quality

Re-running is the whole game: pasted output proves the agent ran commands once (in some environment, at some time); your run proves they pass here, now. Fabrication dies at re-run (the 13-agent lesson); optimism dies too (stale green from pre-change code). Trust the re-run, file the transcript as supporting color.

Step 3 — Adversary-test the forbidden per PR.

# one new test per PR asserting what must NOT happen:
# unpaid export → 403. user B reads user A → null. tampered price → product-row wins.
# name it ADVERSARY_<threat>.test.js — visible, countable, non-skippable in review

Adversary tests are the only tests that prove safety rather than function — everything else asserts the happy path the author already believes. One per PR compounds into a security suite written incident-free (each test names a disaster that never happened because the test exists). Green suites prove features; adversary suites prove sleep.

That's the architecture talking — merges backed by exhibits.

The merge ledger, literature vs exhibits

The real cost, stated: eight minutes per merge plus the social friction of rejecting confident prose ("but it said tested!"). Priced against five incidents and three ruined weekends: proof costs minutes, blind trust costs evenings. Merges are the narrow gate everything flows through — guard the gate and the codebase guards itself. Exhibit A: uninterrupted sleep.

FAQ

Is ai verification gate just CI with extra steps?

CI is half of it (machine re-runs, tireless, dumb). The gate adds human judgment at the right points: designing adversary cases (what must fail), reading proof bundles for completeness (what's missing), and holding the merge pen (accountability lives somewhere). CI automates verification; the gate ritualizes responsibility. Automation without ritual produces green pipelines shipping guesses.

What if the agent genuinely can't produce proof (flaky env, missing data)?

Then the work isn't mergeable yet — say so explicitly ("blocked: needs staging webhook replay") instead of accepting literature. Blocked-with-reason beats merged-on-hope every time; the branch waits cheaply while prod stays safe. Pressure to merge unproven work is how incidents get scheduled. Name the blocker, calendar the unblock, protect main ruthlessly.

Should small docs/copy PRs need the full bundle?

Scaled, never zero: docs get lint plus link-check (two minutes), copy gets preview screenshot (one minute). The principle (evidence over claims) scales down to a breath; only the ceremony shrinks. Skipped-small-PRs teach the pipeline that some merges need no proof — and pipelines generalize exceptions faster than rules. Small proof, always.

How do I review proof bundles without becoming the bottleneck?

Template literacy: reviewers scan four pastes in two minutes (gate exit, happy code, unhappy code, adversary result) versus twenty minutes reading prose. Rejections get faster too (missing paste = instant bounce, no deliberation). Standardized proof compresses review; freeform claims expand it. Bottlenecks come from reading literature, not checking exhibits.

Can proof requirements coexist with fast-moving teams?

They're how fast teams stay fast: unproven merges create incident debt that compounds into velocity-killing firefighting. Teams measuring merge-to-incident (not just merge speed) find proof-gated throughput higher within a quarter. Speed is features shipped per incident, not per day. Gates raise the denominator that matters.

What about agent-to-agent handoffs (no human in loop)?

Same bundle, machine-checked: downstream agent re-runs upstream's pasted commands before building on them (verify, then extend). Chains of unverified agent work amplify hallucinations (the subagents post shows the math); verified chains contain them per link. Every handoff — human or silicon — carries exhibits or it doesn't happen. Trust is per-link, never transitive.

Keep reading

• Agent Said Done. Checkout 500s. (The 10-Second Check) — the corpse this gate autopsies
• Mobile AI Guardrails: Couch Fixes That Can't Break Prod — the gate at phone scale
• AGENTS.md Is a Wish List Until You Add Teeth — the file holding this ritual