[FIX] fool: trial 03 was never the false-positive control, and was inherited as one

Third and largest finding from the trial-03 post-mortem.

The pulling thread — in MEMORY.md and in the previous wrap — named trial 03
'the Fool's false-positive control'. Trial 03's own pre-registration says it
asks whether the checker shares the 2025 archive's self-exemption disposition,
and its grading section states that 'the false-positive rate is still
unmeasured'. The pre-registration knew what it was.

A false-positive control needs a SOUND document, so that 'nothing found' is the
correct answer. Trial 03's input was chosen with five pre-registered weaknesses,
deliberately, because absence of the strong hit is only interpretable if
performance is otherwise competent. The ground-truth list exists to establish
that the document is NOT sound. They are different experiments.

The wrap held the contradiction in one paragraph — calling trial 03 the control
while saying the control requires a sound document trial 03 does not use. It
survived the wake, was restored as the thread, and was 'substrate-checked': the
check verified the M4 was up and that trial 03 had not run, and never asked
whether the trial was the thing the thread said it was. Checking that a claim's
referent exists is not checking that the claim is true. The conflation then
reached the run record's note field, which is preserved with the error in it.

Consequence, larger than trial 03: the false-positive control has not merely
gone unrun, it has never been DESIGNED. It needs a document believed sound, and
soundness cannot be known by construction. That choice is a fork, and it is
surfaced rather than taken.
This commit is contained in:
David F Glidden
2026-08-02 16:52:16 +02:00
parent eda11e559b
commit cd2edaa3c9
2 changed files with 15 additions and 1 deletions
+3 -1
View File
@@ -55,7 +55,9 @@ The v1 Chamber (June–July 2025) ran written work past **two frontier models of
**No false-positive control has ever been run.** Every trial to date used a document with real weaknesses. The claim that the model will say *"nothing found"* on a sound document is **untested** — trial 02's apparent restraint was an artifact of a disabled reasoning mode. Until a clean document is run, the finding-rate cannot be distinguished from a production-rate.
*Still true after trial 03, which was attempted and voided (2026-08-02). Two attempts have now failed for two unrelated reasons, which is itself the finding: the obstacle is not the model but the instrument and the design around it.*
*Still true after trial 03 — and more sharply than "unrun" conveys. **The false-positive control has never been designed, let alone run.** Trial 03 was carried in memory and in the session wrap as "the Fool's false-positive control", but its own pre-registration says it asks whether the checker shares the archive's self-exemption disposition, and its grading section states plainly that "the false-positive rate is still unmeasured". A false-positive control needs a **sound** document so that "nothing found" is the correct answer; trial 03's input was chosen with **five** pre-registered weaknesses precisely so that competence could be verified. The two are different experiments and were conflated in the record.*
*Designing it requires a decision not yet taken: what stands in for a sound document, given that soundness cannot be known? The candidates are a document that has already survived jurist review and steward placement, or a constructed one whose weaknesses are known to be absent. Until that is settled, no trial run can close this item.*
## Grading caveat, standing