Third and largest finding from the trial-03 post-mortem.
The pulling thread — in MEMORY.md and in the previous wrap — named trial 03
'the Fool's false-positive control'. Trial 03's own pre-registration says it
asks whether the checker shares the 2025 archive's self-exemption disposition,
and its grading section states that 'the false-positive rate is still
unmeasured'. The pre-registration knew what it was.
A false-positive control needs a SOUND document, so that 'nothing found' is the
correct answer. Trial 03's input was chosen with five pre-registered weaknesses,
deliberately, because absence of the strong hit is only interpretable if
performance is otherwise competent. The ground-truth list exists to establish
that the document is NOT sound. They are different experiments.
The wrap held the contradiction in one paragraph — calling trial 03 the control
while saying the control requires a sound document trial 03 does not use. It
survived the wake, was restored as the thread, and was 'substrate-checked': the
check verified the M4 was up and that trial 03 had not run, and never asked
whether the trial was the thing the thread said it was. Checking that a claim's
referent exists is not checking that the claim is true. The conflation then
reached the run record's note field, which is preserved with the error in it.
Consequence, larger than trial 03: the false-positive control has not merely
gone unrun, it has never been DESIGNED. It needs a document believed sound, and
soundness cannot be known by construction. That choice is a fork, and it is
surfaced rather than taken.
Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.
Two independent failures, both found by reading the output, neither by a check,
and every check passed:
1. The harness certified a run with no answer. Qwen emitted its scratchpad as
plain prose ('Here's a thinking process:', zero <think> tags), so the tag
regex reported reasoning_present:false and recorded all 2,944 words of
deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
before the answer began. degraded:null. The guard tested the STRING for
emptiness while its field claimed a property of the RESULT — which is the
previous session's open question, answered by the instrument built to audit
instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
harness closed it assuming inline meant tagged.
2. Worse: the design forbade the region it was measuring. The self-exemption
axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
reader to skip author-named limitations, and the scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about
self-reference is indistinguishable from obedience. The axis was unmeasurable
by construction, independent of the truncation. Trial 02's fix and trial 03's
document were each sound alone; their interaction was not.
Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).
The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
David F Glidden