2026-08-29
The benchmark number, measured three times in one day
What we claimed: a detection rate for our benchmark suite. What was wrong: the first two measurements did not survive adversarial audit of the measurement itself: a guard that could fail open, and re-runs that pseudo-replicated the same conditions. What changed: the guard was closed, the matcher tightened, and the number was re-measured. The one we publish since, 50% detection (2 of 4 ground-truth targets, 3 runs, 95% Wilson CI [15%, 85%]), is the first that survived. The two earlier numbers are dead and stay dead.
2026-08-31
The sample report shipped wrong on a live page
What we claimed: a public sample report. What was wrong: the live page showed internal per-run pricing, an internal codename, and copy in the wrong language for the public surface. What changed: same-day fix, followed by an audit of every public file (23 at the time) and a standing pre-deploy guard plus a post-deploy live check, so the class of error, not just the instance, got closed.
2026-09-01
An engagement result our own adjudication refuted
What we claimed: "3 findings delivered, 7 attack classes closed." What was wrong: our own adjudication records, in our own repo, had killed 2 of the 3 findings (one was a false positive, one was a reflection graded as a higher class). What changed: the public claim now reads 1 validated finding, 7 verified-secure closures, and 2 candidates our own adjudication killed before they reached the client. The corrected version is also the more honest story.
2026-09-01
Run-time claims without a measurement behind them
What we claimed: "a full verification run takes ~5-10 minutes." What was wrong: no run-id, no condition, and the number collided with our own sample, which shows 104.5 seconds. Two public artifacts also contradicted each other about what the same run found. What changed: every duration claim now carries its run-id (6292afd9e448, the public sample), and the sample and the benchmark disclosure reconcile explicitly: they are different runs of the same target in different modes, and both say so.
2026-09-01
Online certificate verification that did not exist online
What we claimed: five sentences on the landing pointed to online certificate verification at a URL that served the sample report. What was wrong: the resolver was internal-only; the public claim was ahead of the public software. What changed: all five sentences were rewritten to describe what exists: self-contained certificates whose fingerprints travel with them. The claims go back up only when the public portal ships. The signature public key, also claimed but absent, now lives at a real URL.
2026-09-01
A category-naming claim, refuted by the state of the art
What we claimed: an essay opened by saying the agent-surface category was unnamed and that the essay defined it. What was wrong: OWASP published its Top 10 for Agentic Applications (ASI01 to ASI10) in December 2025, and a deliberately vulnerable MCP server already existed in the OWASP VWAD directory. What changed: the opening now says the category has a name and naming did not prove it about your stack; every surface carries its ASI tag; the prior art is cited; and our contribution is stated as what it actually is, a verdict taxonomy and a proof discipline.
2026-09-01
A cure ordered by narrative, not by evidence
What we claimed: in the GhostJacking essay, data fencing led the four-layer cure. What was wrong: the literature is unambiguous that fencing inside the model is not a security boundary (instructions and data are provably inseparable in shared-embedding architectures), and our own method section already treated hardening as a variable to measure. What changed: the cure is now ordered by strength of evidence: human authorization first, egress second, deterministic scanning third, and fencing last, explicitly labeled hygiene rather than control, which is why we measure it instead of selling it.