David F GliddenandClaude Opus 5
7e19eb51d7
The Fool experiment was not reproducible. Trials 01-02 were run ad hoc: no
script, and of the run conditions only the model ID, MLX version, hardware and
enable_thinking survive. The prompt exists as paraphrase with quoted fragments;
temperature, top_p, max_tokens and seed were never recorded anywhere. Trial 03
could not have been run under trial 02's conditions.
The same failure destroyed the v1 Chamber's GPT-side protocol, discovered today:
it lived as configuration inside a hosted product, was updated in place, and is
gone. The Claude-side prompt from the same morning survives because it was a file
in a repository. A protocol that is not a file is not a protocol.
fool/run_trial.py makes every run a file — prompt hashed into the record, every
sampling parameter recorded including defaults, reasoning trace separated but
never suppressed, and an empty answer marked `degraded` rather than passing as a
finding of silence (trial 02's error, now structurally impossible). Trial 03's
prompt is reconstructed from the surviving fragments and says so in its own
PROVENANCE file: trial 03 is NOT a strict one-variable step from trial 02, and
the chain is clean only from here forward.
ADDENDUM-1 files the measurement the ESCALATE doctrine package states it lacks
("no such measurement exists"). The 2025 Chamber archive, read at steward
direction, shows mutual divergence in 3 of 3 pairs where the instruction was
comparable. Its value is that its parties were of matched capability, so their
divergence cannot be a capability-gap artifact — the arm these trials
structurally cannot produce. Scope held tight: this measures formation
independence between two commercial models. It does NOT answer Q3, the
jurist-executor pair, and the executor's lean there remains none.
Carried as disconfirming evidence: all five interpretive corrections today came
from the steward, not from the executor's own checking, and every one was a
census failure rather than a reading failure. A differently-formed reader of a
document is not positioned to catch those. Formation diversity addresses reading,
not scope.
Nothing applied. The parent package is unmodified; no ratified document edited.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
5.8 KiB
Fool trial log — the running record
The correlation data the differently-biased-checkers doctrine says is owed and has never been produced. Built at n=2 rather than when it becomes a problem — the lesson of the skill-harvest register, which grew to 166 KB before anyone noticed it had stopped being readable. One row per trial. Detail in the per-trial files.
Doctrine under test: differently-biased-checkers-JURIST-PACKAGE-2026-08-01.md (ESCALATE, unruled). Its claim: oversight needs checkers whose contaminations do not point the same way. Its named falsifier: if the parties' misses correlate — if what one misses the others reliably miss too — the principle is false for that configuration.
Standing protocol
- Only documents with known ground truth — packages the jurist has already ruled — so hits and misses are countable.
- Withhold the ruling, the addendum, and any hint of what was found weak.
- Pre-register the grading before the run. Written down, not remembered.
- Prompt gives form, not target. No steer toward any part of the document; explicit anti-contrarian and anti-echo clauses.
enable_thinkingON. Established load-bearing in trial 02 — off produces silence, not brevity.- One variable per trial. Violated in trial 02's first run; the result was uninterpretable and had to be re-run.
- No standing granted to the Fool. Its findings earn a hearing by being checkable, never by role (steward correction, 2026-08-02).
- The protocol is a file, or it is not a protocol. Every trial runs through
fool/run_trial.py, with the prompt as a versioned file hashed into the run record, and every sampling parameter recorded including defaults. Established 2026-08-02 after discovering trials 01–02 are not reproducible — no script, no verbatim prompt, no temperature, top_p, max_tokens or seed. The same failure destroyed the v1 Chamber's GPT-side protocol, which lived as configuration inside a hosted product; the Claude-side prompt from the same day survives because it was a file. - An empty answer is not a finding of silence. If the model emits only a reasoning trace, or nothing, the harness marks the run
degradedand the result may not be graded as restraint. This is trial 02's error made structurally impossible.
Trials
| # | Date | Document | Model | Strong | Moderate | Echo | False-pos | Jurist's central catch |
|---|---|---|---|---|---|---|---|---|
| 01 | 2026-08-01 | PENDING-88 skill-harvest FIX lane | Qwen 3.6 35B-A3B 8bit | MISS | MET ×2 | n/a | untested | missed (narrower test is less safe) |
| 02 | 2026-08-02 | order-attestation (2026-07-29) | Qwen 3.6 35B-A3B 8bit | MISS | MET ×2 | avoided | untested | missed (independence axis) |
What the log shows so far
The correlation the doctrine warns about is present, and localised. 2/2 trials missed the jurist's sharpest contribution. That is not a general blind spot — the Fool produced four real findings across the two trials — it is a specific one, and it now has a shape:
- Jurist — errors of inference: where an argument stopped counting, where an alternative is less safe than it appears.
- Fool — unestablished premises: scope-exhaustiveness, downstream mechanism drift, patterns assumed sound.
- Executor — substrate, containment, arithmetic; and reliably not its own inference errors.
The open question this poses, and it is the sharpest available experiment: is the inference-level miss a property of Qwen, or of any non-jurist reader? A second, differently-formed model run on the same two documents answers it. If it also misses, the gap is structural and no model choice closes it. If it catches, model choice matters far more than assumed.
The 2025 arm — a prior measurement, found not run (2026-08-02)
The v1 Chamber (June–July 2025) ran written work past two frontier models of the moment, preserving both raw outputs unmerged. Read at the steward's direction 2026-08-02; analysed in differently-biased-checkers-ADDENDUM-1-2026-08-02.md.
Mutual divergence in 3 of 3 pairs where the instruction was comparable — each party landing precise textual hits the other missed entirely. The one non-mutual pair is the one whose prompt was most heavily compressed.
Why it matters to this log specifically. These trials run at a large capability gap — a ~35B local model against a frontier one — so divergence here has an alternative explanation: a weaker checker diverging by being weaker rather than by being differently formed. The doctrine is about different bias, not different capability. The 2025 parties were roughly matched, so their divergence cannot be a capability artifact. That is the arm these trials structurally cannot produce, and it returns the same result.
And it supplied trial 03's hypothesis. One 2025 checker exempted the venue it was performing inside, while attacking freely elsewhere, under a system-level instruction reading "No softening." The steward had recorded the same disposition in a user guide dated 2025-01-20: "May smooth over tensions." Trial 03 tests whether ours shares it.
Untested, and load-bearing
No false-positive control has ever been run. Every trial to date used a document with real weaknesses. The claim that the model will say "nothing found" on a sound document is untested — trial 02's apparent restraint was an artifact of a disabled reasoning mode. Until a clean document is run, the finding-rate cannot be distinguished from a production-rate.
Grading caveat, standing
Every grade above was assigned by the executor, whose own errors are among those being graded, and whose reading of what counts as "real" is the reading under test. The findings are individually checkable; the grades are not independent.