Filed in trial 04's tool review, now closed. The guard reported UNTAGGED
SCRATCHPAD ... "Do not grade this as the checker's findings" for both of the two
situations it can see, and they are opposite:
trial 03 — deliberation that ran into the CEILING. No answer ever existed. VOID,
and the absence of findings is NOT restraint.
trial 04 — deliberation that COMPLETED. The answer follows the scratchpad in the
same file. Perfectly gradeable once extracted. NOT void.
Collapsing them would have thrown away six good runs; not distinguishing them
would have graded trial 03's silence as restraint. The guard now branches on
hit_token_ceiling and says which case it is.
Controls added for all four shapes, including the two the trials actually
produced and a clean answer that merely hit the ceiling — truncation is reported
separately and is not a scratchpad problem.
The guard does NOT auto-extract the embedded answer. A heuristic split would be a
new failure mode in the instrument whose entire job is to not silently mis-report
what it has. It flags; a person extracts.
Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.
Two independent failures, both found by reading the output, neither by a check,
and every check passed:
1. The harness certified a run with no answer. Qwen emitted its scratchpad as
plain prose ('Here's a thinking process:', zero <think> tags), so the tag
regex reported reasoning_present:false and recorded all 2,944 words of
deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
before the answer began. degraded:null. The guard tested the STRING for
emptiness while its field claimed a property of the RESULT — which is the
previous session's open question, answered by the instrument built to audit
instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
harness closed it assuming inline meant tagged.
2. Worse: the design forbade the region it was measuring. The self-exemption
axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
reader to skip author-named limitations, and the scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about
self-reference is indistinguishable from obedience. The axis was unmeasurable
by construction, independent of the truncation. Trial 02's fix and trial 03's
document were each sound alone; their interaction was not.
Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).
The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
Two instrument defects, both of the class the harness was built to prevent —
a probe that could not look reporting a value that reads like a result:
- environment() read mlx.__version__, which does not exist (only
mlx.core.__version__). Every run record would have said mlx_version
"unknown" for an installed, versioned package, losing the one field that
makes trial 03 comparable to trial 02. It is MLX 0.31.2, identical.
- git_revision() returns null whenever the harness runs outside its repo,
which is always — it must run on the machine holding the model. The prompt
and input were hashed; the instrument itself was not. Now self-hashed.
The pre-registration addendum is committed BEFORE the run produced output, so
the ordering is checkable rather than asserted. It records: the 'unruled'
premise expiring at REVIEWED-86 (12:13, 32 min after the pre-registration was
written) and why the ordering favours the ground truth; the contamination that
CANNOT be removed, since the amended doctrine is in the executor's auto-loaded
context and I am therefore not a blind grader; the (a)/anti-echo collision
resolved against my own convenience before output existed; and the seed.
Ground truth (a)-(e) is unrevised and will not be revised.
The Fool experiment was not reproducible. Trials 01-02 were run ad hoc: no
script, and of the run conditions only the model ID, MLX version, hardware and
enable_thinking survive. The prompt exists as paraphrase with quoted fragments;
temperature, top_p, max_tokens and seed were never recorded anywhere. Trial 03
could not have been run under trial 02's conditions.
The same failure destroyed the v1 Chamber's GPT-side protocol, discovered today:
it lived as configuration inside a hosted product, was updated in place, and is
gone. The Claude-side prompt from the same morning survives because it was a file
in a repository. A protocol that is not a file is not a protocol.
fool/run_trial.py makes every run a file — prompt hashed into the record, every
sampling parameter recorded including defaults, reasoning trace separated but
never suppressed, and an empty answer marked `degraded` rather than passing as a
finding of silence (trial 02's error, now structurally impossible). Trial 03's
prompt is reconstructed from the surviving fragments and says so in its own
PROVENANCE file: trial 03 is NOT a strict one-variable step from trial 02, and
the chain is clean only from here forward.
ADDENDUM-1 files the measurement the ESCALATE doctrine package states it lacks
("no such measurement exists"). The 2025 Chamber archive, read at steward
direction, shows mutual divergence in 3 of 3 pairs where the instruction was
comparable. Its value is that its parties were of matched capability, so their
divergence cannot be a capability-gap artifact — the arm these trials
structurally cannot produce. Scope held tight: this measures formation
independence between two commercial models. It does NOT answer Q3, the
jurist-executor pair, and the executor's lean there remains none.
Carried as disconfirming evidence: all five interpretive corrections today came
from the steward, not from the executor's own checking, and every one was a
census failure rather than a reading failure. A differently-formed reader of a
document is not positioned to catch those. Formation diversity addresses reading,
not scope.
Nothing applied. The parent package is unmodified; no ratified document edited.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
David F Glidden