[FIX] fool trial 03 VOID; degraded-guard rebuilt with a positive control

Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.

Two independent failures, both found by reading the output, neither by a check,
and every check passed:

1. The harness certified a run with no answer. Qwen emitted its scratchpad as
   plain prose ('Here's a thinking process:', zero <think> tags), so the tag
   regex reported reasoning_present:false and recorded all 2,944 words of
   deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
   before the answer began. degraded:null. The guard tested the STRING for
   emptiness while its field claimed a property of the RESULT — which is the
   previous session's open question, answered by the instrument built to audit
   instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
   harness closed it assuming inline meant tagged.

2. Worse: the design forbade the region it was measuring. The self-exemption
   axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
   reader to skip author-named limitations, and the scratchpad shows the model
   reaching Part VII and leaving it, citing that constraint. Silence about
   self-reference is indistinguishable from obedience. The axis was unmeasurable
   by construction, independent of the truncation. Trial 02's fix and trial 03's
   document were each sound alone; their interaction was not.

Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).

The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
This commit is contained in:
David F Glidden
2026-08-02 16:50:59 +02:00
parent b678d2f57b
commit eda11e559b
8 changed files with 608 additions and 15 deletions
@@ -73,9 +73,15 @@ grade will not be independent.
## Addendum, written DURING the run and BEFORE any output was seen
*(Run launched 2026-08-02 ~13:0x; model still loading; the output file was empty when
each item below was written. Recorded here rather than in the write-up precisely because
its whole value is that it precedes the result.)*
*(Run launched 2026-08-02 mid-afternoon; the model was still loading and the output file was
verifiably empty when each item below was written. Recorded here rather than in the write-up
precisely because its whole value is that it precedes the result — so the claim is committed
as `b678d2f`, whose timestamp is checkable, rather than asserted in prose. The run's own
`started_utc` in the run record is the other half of the ordering.*
*A wrong clock-time — "~13:0x" — stood in this line in `b678d2f`. It was four hours off, in a
document whose entire load-bearing property is its timestamps. Corrected here rather than
quietly, because the correction is the sort of thing this file exists to make visible.)*
**1. The "unruled" premise above expired 32 minutes after it was written.** It was true at
11:41. At **12:13** the steward placed **REVIEWED-86**, design-gating this doctrine with two
@@ -116,6 +122,40 @@ Second: the harness now hashes *itself* into the run record, because `git_revisi
null whenever the harness runs outside its repository — which is always, since it must run on
the machine holding the model.
## Result
## Result — written AFTER the run, and marked as such
*(To be filled after the run. Empty until then — deliberately.)*
**VOID.** Not STRONG HIT, not EXEMPTION SIGNAL, not NULL, not ECHO. The trial did not
produce a gradeable output, and its axis could not have been measured even if it had.
Full write-up: `../fool-trial-03-2026-08-02.md`. Run record:
`runs/trial-03-20260802T144136Z.*`.
1. **No answer was produced.** Qwen emitted an untagged scratchpad and exhausted the
4,096-token ceiling before beginning its answer. The harness recorded
`degraded: null` — it tested the string for emptiness while the field claimed the
result was sound. Fixed this session, with a positive control that runs against the
actual artefact (`test_degraded_guard.py`).
2. **The axis was unmeasurable by construction, and this is the design's fault, not the
run's.** The self-exemption signal lives in Part VII; the prompt's anti-echo constraint
tells the reader to skip author-named limitations. The scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about self-reference
is therefore indistinguishable from obedience.
**The pre-registration above did not catch this, and the reason is worth recording:
it reasoned about the document and about the grading, and never about the prompt
already sitting in the file.** The PROVENANCE note warned that the prompt was
reconstructed and that trial 03 was not a one-variable step — and the warning was
read as a caveat on *comparability* rather than as a reason to re-read what the
prompt instructs. The ladder's own rule covers it: re-run verification at the scope
of the extension.
**Ground truth (a)–(e) was not revised, and was not scored** — there is no valid output to
score. The comparison against REVIEWED-86 is therefore **not performed**; it waits for a
valid run.
**The pre-run addendum earned its keep.** Every item in it held up, and item 2 — the
(a)/anti-echo collision, resolved against my own convenience before output existed — was
the thread that led to failure 2. Having already ruled that the anti-echo clause excluded
part of ground truth (a), the question *what else does it exclude?* was available. It was
not asked until the output forced it, which is the honest limit on how much credit the
addendum deserves.