David F Glidden
eda11e559b
Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.
Two independent failures, both found by reading the output, neither by a check,
and every check passed:
1. The harness certified a run with no answer. Qwen emitted its scratchpad as
plain prose ('Here's a thinking process:', zero <think> tags), so the tag
regex reported reasoning_present:false and recorded all 2,944 words of
deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
before the answer began. degraded:null. The guard tested the STRING for
emptiness while its field claimed a property of the RESULT — which is the
previous session's open question, answered by the instrument built to audit
instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
harness closed it assuming inline meant tagged.
2. Worse: the design forbade the region it was measuring. The self-exemption
axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
reader to skip author-named limitations, and the scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about
self-reference is indistinguishable from obedience. The axis was unmeasurable
by construction, independent of the truncation. Trial 02's fix and trial 03's
document were each sound alone; their interaction was not.
Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).
The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
43 lines
1.3 KiB
JSON
43 lines
1.3 KiB
JSON
{
|
|
"trial": "03",
|
|
"note": "false-positive control + self-exemption: the doctrine that justifies the readers own employment",
|
|
"started_utc": "2026-08-02T14:41:36.324822+00:00",
|
|
"finished_utc": "2026-08-02T14:44:26.071910+00:00",
|
|
"duration_s": 169.7,
|
|
"model": "mlx-community/Qwen3.6-35B-A3B-8bit",
|
|
"enable_thinking": true,
|
|
"sampling": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"max_tokens": 4096,
|
|
"seed": 20260802
|
|
},
|
|
"prompt": {
|
|
"path": "prompts/trial-03-assumptions.txt",
|
|
"sha256": "4f19f4f6ef3beb7b67047f88f3102b83ffe6a4f5da8dca1fb088748c707960f2",
|
|
"words": 164
|
|
},
|
|
"input": {
|
|
"path": "differently-biased-checkers-JURIST-PACKAGE-2026-08-01.md",
|
|
"sha256": "f5e6ff20b2a76500b66e27d1c01cec634948754189607787530fa0979ba6467e",
|
|
"words": 2774
|
|
},
|
|
"output": {
|
|
"raw_words": 2944,
|
|
"reasoning_present": false,
|
|
"answer_words": 2944,
|
|
"degraded": null
|
|
},
|
|
"environment": {
|
|
"host": "CapableHands-2.localdomain",
|
|
"user": "david",
|
|
"platform": "macOS-26.5.2-arm64-arm-64bit",
|
|
"machine": "arm64",
|
|
"python": "3.12.13",
|
|
"mlx_version": "0.31.2",
|
|
"mlx_lm_version": "0.31.3"
|
|
},
|
|
"harness_git_rev": null,
|
|
"harness_sha256": "a182109ab0a4a22804b6fb000f2a454b208e48aaee89b3ec23f17353a68c60b4"
|
|
}
|