When our own claims fail re-measurement, we publish the amendment. This page lists every public claim we have withdrawn or corrected, with dates. A company that grades graders has to keep this page boring.
2026-08-29
The benchmark number, measured three times in one day
What we claimed: a detection rate for our benchmark suite. What was wrong: the first two measurements did not survive adversarial audit of the measurement itself: a guard that could fail open, and re-runs that pseudo-replicated the same conditions. What changed: the guard was closed, the matcher tightened, and the number was re-measured. The one we publish since, 50% detection (2 of 4 ground-truth targets, 3 runs, 95% Wilson CI [15%, 85%]), is the first that survived. The two earlier numbers are dead and stay dead.
2026-08-31
The sample report shipped wrong on a live page
What we claimed: a public sample report. What was wrong: the live page showed internal per-run pricing, an internal codename, and copy in the wrong language for the public surface. What changed: same-day fix, followed by an audit of every public file (23 at the time) and a standing pre-deploy guard plus a post-deploy live check, so the class of error, not just the instance, got closed.
2026-09-01
An engagement result our own adjudication refuted
What we claimed: "3 findings delivered, 7 attack classes closed." What was wrong: our own adjudication records, in our own repo, killed 2 of the 3 headline candidates: the CRITICAL was a public endpoint behaving as designed, and the SSRF-HIGH was a reflection our attribution rule refused to count as an outbound hit. The delivered engagement was 1 validated finding plus 7 verified-resistances closures (the two kills came from different runs of the engagement, and the count we first published blurred that). What changed: the public claim now reads 1 validated finding, 7 verified-resistances closures, and 2 candidates our own adjudication killed before they reached the client. The corrected version is also the more honest story.
[2026-09-03] Acknowledgment this entry has owed since September 2: when it was first corrected, it was rewritten in place, without a dated note — a silent edit on the page whose job is to forbid them. The entry above stands as history and is not edited again; later reconciliations land as appended notes like this one. The refund followed our correction policy (re-issue plus refund of the excess over delivered value — or the full price at the client's choice).
2026-09-01
Run-time claims without a measurement behind them
What we claimed: "a full verification run takes ~5-10 minutes." What was wrong: no run-id, no condition, and the number collided with our own sample, which shows 104.5 seconds. Two public artifacts also contradicted each other about what the same run found. What changed: every duration claim now carries its run-id (6292afd9e448, the public sample), and the sample and the benchmark disclosure reconcile explicitly: they are different runs of the same target in different modes, and both say so.
[2026-09-05] The 104.5 s / run 6292afd9e448 anchor refers to the retired sample; the current sample is run 7c30ed04dc73 (see the entry below).
2026-09-01
Online certificate verification that did not exist online
What we claimed: five sentences on the landing pointed to online certificate verification at a URL that served the sample report. What was wrong: the resolver was internal-only; the public claim was ahead of the public software. What changed: all five sentences were rewritten to describe what exists: self-contained certificates whose fingerprints travel with them. The claims go back up only when the public portal ships. The signature public key, also claimed but absent, now lives at a real URL.
2026-09-01
A category-naming claim, refuted by the state of the art
What we claimed: an essay opened by saying the agent-surface category was unnamed and that the essay defined it. What was wrong: OWASP published its Top 10 for Agentic Applications (ASI01 to ASI10) in December 2025, and a deliberately vulnerable MCP server already existed in the OWASP VWAD directory. What changed: the opening now says the category has a name and naming did not prove it about your stack; every surface carries its ASI tag; the prior art is cited; and our contribution is stated as what it actually is, a verdict taxonomy and a proof discipline.
2026-09-01
A cure ordered by narrative, not by evidence
What we claimed: in the GhostJacking essay, data fencing led the four-layer cure. What was wrong: the literature is unambiguous that fencing inside the model is not a security boundary (instructions and data are provably inseparable in shared-embedding architectures), and our own method section already treated hardening as a variable to measure. What changed: the cure is now ordered by strength of evidence: human authorization first, egress second, deterministic scanning third, and fencing last, explicitly labeled hygiene rather than control, which is why we measure it instead of selling it.
2026-09-02
This page, corrected by its own standard
What was wrong: entry 4 claimed a reconciliation ("both say so") before it was live on the home page; entry 3 was rewritten in place without a note; and the language guard documented on Aug 31 did not cover two artifacts it later caught (the MCP matrix and the public key filename). What changed: the reconciliation sentence shipped, entry 3 now carries this acknowledgment instead of silent edits, and the guard's coverage was extended the same day. A corrections page that edits itself silently would be the one thing it cannot be.
2026-09-03
The public sample was replaced, and the old one's SSRF is now refused, not downgraded
What we claimed: a public sample report anchored to run 6292afd9e448 (2026-08-26, 104.5 s, 5 findings). What was wrong: the run predated the validation door we ship behind today — the reflection guard on SSRF, the class check against public-by-design endpoints, and adversarial adjudication all landed after Aug 26, so reflection-based SSRF and public-by-design auth_bypass no longer credit as findings. The old sample kept citing graded-by-the-old-door numbers as if they were current. What changed: the sample was regenerated from a current full-capacity run and the old one was retired. Endpoint by endpoint: the SSRF-HIGH 8.6 on /redirect?to was attacked again today — 8 attempts across 3 informed rounds — and is not credited at any severity, because the attribution rule refuses to count our own reflection as an outbound hit (refused, not downgraded); the two BOLA the old run credited were attacked 8-11 times each across the same informed rounds without firing, and are declared attacked-not-fired — a resistance verdict, not new security. The current sample is run 7c30ed04dc73 (2026-09-03, 374.1 s): 11 hypotheses raised, 2 CRITICAL SQLi findings validated, 1 candidate excluded by adjudication with the reason persisted, and 8 attack classes attacked-not-fired. The published artifact is the sample report, whose report JSON ships at sha256 33ad51abeeb5085b10003848e6606fff411a9c82e06a5f86d23a8c694aac1c11, redacted for public release only in client references inherited from cross-target memory, cost fields, and operational parameters — every finding, PoC, and metric is verbatim from the run.
2026-09-03
A human-approval claim, narrowed to what we actually gate
What we claimed: "A human operator approves every action." What was wrong: it over-claimed. Batch pipeline runs — the run itself, report generation, retries — execute under the signed scope; per-action gating applies to agent actions during live operation. The narrowing shipped with the September 3 rebuild of the home page without an entry here on the day, which is the pattern this page exists to catch. What changed: four sentences now read "agent actions during live operation pass a human gate" — in the how-it-works section, in the 48-hour note, and in two FAQ answers — and this entry registers the change after the fact.
[2026-09-05] Same week, per-reason classification was added: of those eight non-findings, 5 held under attack, 2 were adjudicated out (one reclassified, one validated-then-refuted on reflection) and 2 were inconclusive (ambiguous signal, reflection). The current sample and certificate show the four states; nothing that was adjudicated or inconclusive is counted as resistance.
2026-09-06
The published sample's remediation guidance carried a technically wrong recommendation — amended, original preserved
The public run artifact's remediation for the login finding suggested enabling SQLite WAL as a SQLi defense. WAL is a journaling mechanism, not an injection mitigation, and the accompanying snippet omitted password verification. A technical amendment corrects the guidance; the original artifact is preserved; the artifact is preserved byte-identical (sha256 unchanged) as the historical record, and this entry links both.