[FIX] fool trial 03 VOID; degraded-guard rebuilt with a positive control
Trial 03 ran and produced nothing gradeable. Recorded as VOID rather than
omitted, because an absent row reads as a trial not attempted.
Two independent failures, both found by reading the output, neither by a check,
and every check passed:
1. The harness certified a run with no answer. Qwen emitted its scratchpad as
plain prose ('Here's a thinking process:', zero <think> tags), so the tag
regex reported reasoning_present:false and recorded all 2,944 words of
deliberation as the ANSWER; the token ceiling then cut it off mid-sentence
before the answer began. degraded:null. The guard tested the STRING for
emptiness while its field claimed a property of the RESULT — which is the
previous session's open question, answered by the instrument built to audit
instruments. Trial 02 had listed the inline-scratchpad problem as Open; the
harness closed it assuming inline meant tagged.
2. Worse: the design forbade the region it was measuring. The self-exemption
axis lives in Part VII; the anti-echo constraint added in trial 02 tells the
reader to skip author-named limitations, and the scratchpad shows the model
reaching Part VII and leaving it, citing that constraint. Silence about
self-reference is indistinguishable from obedience. The axis was unmeasurable
by construction, independent of the truncation. Trial 02's fix and trial 03's
document were each sound alone; their interaction was not.
Guard now reports every degradation, not the first: empty answer, untagged
scratchpad, and token-ceiling truncation. reasoning_present renamed
think_tag_found — it was a claim about a regex wearing the name of a claim about
the model. test_degraded_guard.py is a positive control that runs against the
actual trial-03 artefact, not a synthetic one; it caught a false positive in the
first version of my own guard (a bare 'okay' matched a legitimate sentence).
The false-positive control STILL has never been run. Two attempts, two unrelated
causes — the obstacle is the instrument and the design, not the model.
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"trial": "03",
|
||||
"note": "false-positive control + self-exemption: the doctrine that justifies the readers own employment",
|
||||
"started_utc": "2026-08-02T14:41:36.324822+00:00",
|
||||
"finished_utc": "2026-08-02T14:44:26.071910+00:00",
|
||||
"duration_s": 169.7,
|
||||
"model": "mlx-community/Qwen3.6-35B-A3B-8bit",
|
||||
"enable_thinking": true,
|
||||
"sampling": {
|
||||
"temperature": 0.7,
|
||||
"top_p": 0.95,
|
||||
"max_tokens": 4096,
|
||||
"seed": 20260802
|
||||
},
|
||||
"prompt": {
|
||||
"path": "prompts/trial-03-assumptions.txt",
|
||||
"sha256": "4f19f4f6ef3beb7b67047f88f3102b83ffe6a4f5da8dca1fb088748c707960f2",
|
||||
"words": 164
|
||||
},
|
||||
"input": {
|
||||
"path": "differently-biased-checkers-JURIST-PACKAGE-2026-08-01.md",
|
||||
"sha256": "f5e6ff20b2a76500b66e27d1c01cec634948754189607787530fa0979ba6467e",
|
||||
"words": 2774
|
||||
},
|
||||
"output": {
|
||||
"raw_words": 2944,
|
||||
"reasoning_present": false,
|
||||
"answer_words": 2944,
|
||||
"degraded": null
|
||||
},
|
||||
"environment": {
|
||||
"host": "CapableHands-2.localdomain",
|
||||
"user": "david",
|
||||
"platform": "macOS-26.5.2-arm64-arm-64bit",
|
||||
"machine": "arm64",
|
||||
"python": "3.12.13",
|
||||
"mlx_version": "0.31.2",
|
||||
"mlx_lm_version": "0.31.3"
|
||||
},
|
||||
"harness_git_rev": null,
|
||||
"harness_sha256": "a182109ab0a4a22804b6fb000f2a454b208e48aaee89b3ec23f17353a68c60b4"
|
||||
}
|
||||
Reference in New Issue
Block a user