[FIX] fool: trial 03 was never the false-positive control, and was inherited as one

Third and largest finding from the trial-03 post-mortem.

The pulling thread — in MEMORY.md and in the previous wrap — named trial 03
'the Fool's false-positive control'. Trial 03's own pre-registration says it
asks whether the checker shares the 2025 archive's self-exemption disposition,
and its grading section states that 'the false-positive rate is still
unmeasured'. The pre-registration knew what it was.

A false-positive control needs a SOUND document, so that 'nothing found' is the
correct answer. Trial 03's input was chosen with five pre-registered weaknesses,
deliberately, because absence of the strong hit is only interpretable if
performance is otherwise competent. The ground-truth list exists to establish
that the document is NOT sound. They are different experiments.

The wrap held the contradiction in one paragraph — calling trial 03 the control
while saying the control requires a sound document trial 03 does not use. It
survived the wake, was restored as the thread, and was 'substrate-checked': the
check verified the M4 was up and that trial 03 had not run, and never asked
whether the trial was the thing the thread said it was. Checking that a claim's
referent exists is not checking that the claim is true. The conflation then
reached the run record's note field, which is preserved with the error in it.

Consequence, larger than trial 03: the false-positive control has not merely
gone unrun, it has never been DESIGNED. It needs a document believed sound, and
soundness cannot be known by construction. That choice is a fork, and it is
surfaced rather than taken.
This commit is contained in:
David F Glidden
2026-08-02 16:52:16 +02:00
parent eda11e559b
commit cd2edaa3c9
2 changed files with 15 additions and 1 deletions
@@ -73,6 +73,18 @@ Per the pre-run addendum: I am **not a blind grader**. The doctrine as amended u
---
## Failure 3 — trial 03 was never the false-positive control, and was inherited as one
The pulling thread carried into this session, from the previous wrap and from `MEMORY.md`, reads: *"**PULLING THREAD: Trial 03 — the Fool's false-positive control.** … It is the one measurement that can distinguish a *finding-rate* from a *production-rate* — until a **sound** document is run…"*
**Trial 03's pre-registered design is not that experiment.** Its own text says *"Trial 03 asks whether our checker shares it"* — *it* being the targeted self-exemption found in the 2025 archive. And its grading section says, of the false-positive category, *"the false-positive rate is still unmeasured."* The pre-registration knew what it was and said so.
A false-positive control requires a **sound** document — one with no real weaknesses — so that *"nothing found"* is the correct answer. The input here is the opposite: I pre-registered **five** genuine weaknesses in it, deliberately, because *"absence of the strong hit is only interpretable if performance is otherwise competent."* The ground-truth list exists precisely to establish that the document is **not** sound.
So the wrap contained the contradiction in a single paragraph — naming trial 03 the false-positive control while stating that the control requires a sound document trial 03 does not use — and it survived being read at the wake, restored as the thread, and confirmed by a substrate check that verified the M4 was reachable and that trial 03 had not run, and never asked whether the trial was the thing the thread said it was. **I then propagated the conflation into the run record itself**, whose `note` field now reads `"false-positive control + self-exemption"`. The record is preserved with the error in it.
**Consequence for the programme, which is larger than trial 03.** The false-positive control has not merely gone unrun — **it has never been designed.** It needs a document believed sound, and by construction we cannot know a document is sound; the best available approximations are a document that has already survived jurist review and steward placement, or a deliberately constructed one whose weaknesses are known to be absent. That choice has not been made. Two attempts at "the control" have now failed, and neither was an attempt at the control.
## What changes before trial 04
1. **`degraded` must fire on more than emptiness** — untagged scratchpad detected, and generation stopped at the token ceiling. *(Fixed in the harness this session; see below.)*
+3 -1
View File
@@ -55,7 +55,9 @@ The v1 Chamber (June–July 2025) ran written work past **two frontier models of
**No false-positive control has ever been run.** Every trial to date used a document with real weaknesses. The claim that the model will say *"nothing found"* on a sound document is **untested** — trial 02's apparent restraint was an artifact of a disabled reasoning mode. Until a clean document is run, the finding-rate cannot be distinguished from a production-rate.
*Still true after trial 03, which was attempted and voided (2026-08-02). Two attempts have now failed for two unrelated reasons, which is itself the finding: the obstacle is not the model but the instrument and the design around it.*
*Still true after trial 03 — and more sharply than "unrun" conveys. **The false-positive control has never been designed, let alone run.** Trial 03 was carried in memory and in the session wrap as "the Fool's false-positive control", but its own pre-registration says it asks whether the checker shares the archive's self-exemption disposition, and its grading section states plainly that "the false-positive rate is still unmeasured". A false-positive control needs a **sound** document so that "nothing found" is the correct answer; trial 03's input was chosen with **five** pre-registered weaknesses precisely so that competence could be verified. The two are different experiments and were conflated in the record.*
*Designing it requires a decision not yet taken: what stands in for a sound document, given that soundness cannot be known? The candidates are a document that has already survived jurist review and steward placement, or a constructed one whose weaknesses are known to be absent. Until that is settled, no trial run can close this item.*
## Grading caveat, standing