87f577a2c816dadb5e92c995f0b5ef8f1b8a00c0
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 David F Glidden
|
eda11e559b |
[FIX] fool trial 03 VOID; degraded-guard rebuilt with a positive control
Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.
Two independent failures, both found by reading the output, neither by a check,
and every check passed:
1. The harness certified a run with no answer. Qwen emitted its scratchpad as
plain prose ('Here's a thinking process:', zero <think> tags), so the tag
regex reported reasoning_present:false and recorded all 2,944 words of
deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
before the answer began. degraded:null. The guard tested the STRING for
emptiness while its field claimed a property of the RESULT — which is the
previous session's open question, answered by the instrument built to audit
instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
harness closed it assuming inline meant tagged.
2. Worse: the design forbade the region it was measuring. The self-exemption
axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
reader to skip author-named limitations, and the scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about
self-reference is indistinguishable from obedience. The axis was unmeasurable
by construction, independent of the truncation. Trial 02's fix and trial 03's
document were each sound alone; their interaction was not.
Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).
The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
|
||
|
David F Glidden
|
b678d2f57b |
[FIX] fool harness: record mlx version correctly + self-hash; trial-03 pre-run addendum
Two instrument defects, both of the class the harness was built to prevent — a probe that could not look reporting a value that reads like a result: - environment() read mlx.__version__, which does not exist (only mlx.core.__version__). Every run record would have said mlx_version "unknown" for an installed, versioned package, losing the one field that makes trial 03 comparable to trial 02. It is MLX 0.31.2, identical. - git_revision() returns null whenever the harness runs outside its repo, which is always — it must run on the machine holding the model. The prompt and input were hashed; the instrument itself was not. Now self-hashed. The pre-registration addendum is committed BEFORE the run produced output, so the ordering is checkable rather than asserted. It records: the 'unruled' premise expiring at REVIEWED-86 (12:13, 32 min after the pre-registration was written) and why the ordering favours the ground truth; the contamination that CANNOT be removed, since the amended doctrine is in the executor's auto-loaded context and I am therefore not a blind grader; the (a)/anti-echo collision resolved against my own convenience before output existed; and the seed. Ground truth (a)-(e) is unrevised and will not be revised. |
||
|
David F GliddenandClaude Opus 5
|
bdf24c044b |
[FIX] Addendum-1: make the central claim checkable; add containment proof
Two defects in the addendum as first filed, both found by checking rather than by reading. First, it asserted a set comparison over documents the jurist cannot read. Its own header promises every clause reasoned about is quoted verbatim, but the claim the addendum rests on -- mutual divergence in 3 of 3 comparable pairs -- was a summary of the executor's own analysis. The appendix now reproduces one pair as an eleven-row side-by-side of extracted claims, verbatim where quoted, so the comparison can be checked independently. The pair chosen is the least confounded rather than the most favourable: the v1 standard prompt is model-agnostic and needs no compressed variant, so both parties demonstrably read the same file. What the jurist still cannot check is stated explicitly. Second, Part E rendered a bullet list from the 2025-01-20 source as running prose with terminal periods the source does not contain, inside a blockquote. A blockquote asserts verbatim. Same family as the truncation that closed a sentence with an invented word on 2026-08-01, and again caught mechanically. Corrected in all three files where it appeared; the fabricated period is now a positive control, so the instrument proves it catches this defect. check_containment.py generalises the check that found it. Positive controls are mandatory -- it exits non-zero if none are declared, because a check reporting all-pass without them cannot be distinguished from one unable to detect absence. Addendum-1 now carries its result: 28/28 contained, 5/5 controls absent. Not filed as satisfying PENDING-86 option (b), which is unruled and concerns whether such a proof should be REQUIRED of every package. This is the executor checking its own work before filing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc |
||
|
David F GliddenandClaude Opus 5
|
7e19eb51d7 |
[FIX] Fool: make trials reproducible; file the 2025 correlation measurement
The Fool experiment was not reproducible. Trials 01-02 were run ad hoc: no
script, and of the run conditions only the model ID, MLX version, hardware and
enable_thinking survive. The prompt exists as paraphrase with quoted fragments;
temperature, top_p, max_tokens and seed were never recorded anywhere. Trial 03
could not have been run under trial 02's conditions.
The same failure destroyed the v1 Chamber's GPT-side protocol, discovered today:
it lived as configuration inside a hosted product, was updated in place, and is
gone. The Claude-side prompt from the same morning survives because it was a file
in a repository. A protocol that is not a file is not a protocol.
fool/run_trial.py makes every run a file — prompt hashed into the record, every
sampling parameter recorded including defaults, reasoning trace separated but
never suppressed, and an empty answer marked `degraded` rather than passing as a
finding of silence (trial 02's error, now structurally impossible). Trial 03's
prompt is reconstructed from the surviving fragments and says so in its own
PROVENANCE file: trial 03 is NOT a strict one-variable step from trial 02, and
the chain is clean only from here forward.
ADDENDUM-1 files the measurement the ESCALATE doctrine package states it lacks
("no such measurement exists"). The 2025 Chamber archive, read at steward
direction, shows mutual divergence in 3 of 3 pairs where the instruction was
comparable. Its value is that its parties were of matched capability, so their
divergence cannot be a capability-gap artifact — the arm these trials
structurally cannot produce. Scope held tight: this measures formation
independence between two commercial models. It does NOT answer Q3, the
jurist-executor pair, and the executor's lean there remains none.
Carried as disconfirming evidence: all five interpretive corrections today came
from the steward, not from the executor's own checking, and every one was a
census failure rather than a reading failure. A differently-formed reader of a
document is not positioned to catch those. Formation diversity addresses reading,
not scope.
Nothing applied. The parent package is unmodified; no ratified document edited.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
|