David F Glidden
4408506ffa
Prediction recorded in Reduction 01 BEFORE this census, so it could fail: the
package's Part I is 'Grounding (quoted verbatim)' and quotes CLAUDE.md directly,
so Q should be non-zero where it was zero. Q=9. D=40, where the ruling had none.
ruling package
sound 8.5% 68.6%
PERFORMATIVE 12 0 <- the genre signature
BLEND 9 25
INHERITED 4 0
UNSOURCED-QUOTE 3 0 <- §1's header clause worked
Genre reading confirmed eightfold: a package proposes, a ruling determines.
CORRECTS Reduction 01's strong conclusion that 'the reduction arm collapses into
the synthetic arm'. On package prose repair touches 31.4%, not 91.5% — reduction,
not authoring, and the two arms stay distinct. That conclusion was correctly
bounded at n=1; the bound was the whole of its content and one document collapsed
it. Reduction 01 now carries the correction inline.
BLEND is now the blocker and is genre-independent: 25 of 33 quarantines, 7 of
them rows of the Part IV table, which pairs a quote with an end-state and a
verdict — three primitives by construction.
Two check findings, one good and one bad:
§3.2 CAUGHT A REAL TAGGING ERROR OF MINE. Unit 145 was tagged Q; it is a sentence
ABOUT a quotation, not a quotation, so not verbatim-as-a-unit. Corrected to D.
The check found it, the reading did not — the 'quoted but not traced' defect the
jurist caught on 2026-07-19, mechanised.
§3.3 GAVE A FALSE PASS, found by looking. Part VII 'Disconfirming evidence' IS a
collected limitations section under §2a — the exact section trial 03 showed the
model skipping wholesale — and the screen missed it because it never says
'limitations'. Widened; the package now correctly FAILS §3.3. But no pattern can
decide this: a section titled only 'Part VII' defeats any wordlist, and a control
now asserts that. §3.3 is a SCREEN, not a decision; §2a belongs in §4's judgement
residue. Fourth time in three days a passing check certified the code while the
property failed, and the fourth found by a person looking.
A=0 IN BOTH DOCUMENTS, and it is the same fact as the §2a failure seen from the
other side: we do not name assumptions inline, we collect them into a section.
Our best governance prose is written in exactly the shape that defeats the reader
the section was written for.
Also fixed: the tool was still printing 'NOT checked here: §3.2' after §3.2 was
implemented — under-claiming, but still a false statement about what ran.
Kernel v1.1 candidates are now evidence-backed and remain UNAPPLIED; v1.0 stays
frozen and a revision is a new experiment. The false-positive control remains
unrun and neither reduction produced a usable control document.
259 lines
8.8 KiB
Python
Executable File
259 lines
8.8 KiB
Python
Executable File
#!/usr/bin/env python3
|
||
"""
|
||
Positive controls for the reduction tooling.
|
||
|
||
Control Kernel v1.0 §3: "Each check ships with a positive control — a fixture it
|
||
is shown to fail on — before any result from it is believed. An absence is not
|
||
evidence until the instrument is shown capable of detecting presence."
|
||
|
||
So every gate below is shown FAILING on a fixture built to break it, and passing
|
||
on one built not to. A gate only ever demonstrated passing has demonstrated
|
||
nothing.
|
||
|
||
Usage: ./test_reduce.py
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||
from reduce import ( # noqa: E402
|
||
FORBIDDEN_HEADING_RE,
|
||
TAGGABLE,
|
||
split_spans,
|
||
verify_tiling,
|
||
)
|
||
|
||
failures: list[str] = []
|
||
|
||
|
||
def check(name: str, got, want, detail: str = "") -> None:
|
||
if got != want:
|
||
failures.append(f"{name}: expected {want!r}, got {got!r}. {detail}")
|
||
print(f" FAIL {name}")
|
||
else:
|
||
print(f" ok {name}")
|
||
|
||
|
||
SAMPLE = """# A heading
|
||
|
||
Some prose here. It has two sentences.
|
||
|
||
- a list item
|
||
- another
|
||
|
||
> a quoted block
|
||
|
||
```
|
||
code that must not be split. really.
|
||
```
|
||
|
||
Final paragraph, e.g. with an abbreviation inside it. And a second sentence.
|
||
"""
|
||
|
||
print("Tiling invariant — the gate everything else depends on:")
|
||
spans = split_spans(SAMPLE)
|
||
check("sample tiles cleanly", verify_tiling(spans, SAMPLE), [])
|
||
check(
|
||
"spans reproduce source byte-for-byte",
|
||
"".join(SAMPLE[s["start"]:s["end"]] for s in spans),
|
||
SAMPLE,
|
||
)
|
||
|
||
print("\nPositive control — the gate must DETECT a broken tiling:")
|
||
gap = [dict(s) for s in spans]
|
||
gap[2]["start"] += 1 # open a one-byte hole
|
||
check("gap detected", len(verify_tiling(gap, SAMPLE)) > 0, True, "gate blind to a gap")
|
||
|
||
overlap = [dict(s) for s in spans]
|
||
overlap[2]["start"] -= 1 # overlap the previous span
|
||
check("overlap detected", len(verify_tiling(overlap, SAMPLE)) > 0, True)
|
||
|
||
truncated = [dict(s) for s in spans[:-1]]
|
||
check("truncation detected", len(verify_tiling(truncated, SAMPLE)) > 0, True)
|
||
|
||
print("\nSplitter behaviour:")
|
||
prose = [s for s in spans if s["kind"] == "prose"]
|
||
texts = [SAMPLE[s["start"]:s["end"]] for s in prose]
|
||
check("abbreviation did not split 'e.g.'", sum("e.g." in t for t in texts), 1)
|
||
check(
|
||
"'e.g.' sentence not broken after the abbreviation",
|
||
any(t.strip().startswith("Final paragraph, e.g. with") for t in texts),
|
||
True,
|
||
f"prose units: {texts}",
|
||
)
|
||
check("two sentences found in para 1", sum("Some prose here." in t for t in texts), 1)
|
||
check(
|
||
"code fence never becomes prose",
|
||
any("code that must not be split" in SAMPLE[s["start"]:s["end"]] and s["kind"] == "code"
|
||
for s in spans),
|
||
True,
|
||
)
|
||
check(
|
||
"list items are not prose",
|
||
all("a list item" not in t for t in texts),
|
||
True,
|
||
)
|
||
check("headings are taggable", "heading" in TAGGABLE, True,
|
||
"kernel §4 rules a heading can assert")
|
||
|
||
print("\nDefects found by contact with a real ruling (splitter v1.1.0):")
|
||
|
||
FENCED = """Ready to paste:
|
||
|
||
```
|
||
## REVIEWED-85 — a heading INSIDE a fence
|
||
**Date:** 2026-08-01
|
||
```
|
||
|
||
After the fence.
|
||
"""
|
||
fspans = split_spans(FENCED)
|
||
check("fenced doc tiles", verify_tiling(fspans, FENCED), [])
|
||
check(
|
||
"(a) '##' inside a fence is code, not a taggable heading",
|
||
any(s["kind"] == "heading" and "REVIEWED-85" in FENCED[s["start"]:s["end"]]
|
||
for s in fspans),
|
||
False,
|
||
"quoted content must not become structure of the quoting document",
|
||
)
|
||
|
||
RULE = "Some prose.\n\n---\n\nMore prose.\n"
|
||
rspans2 = split_spans(RULE)
|
||
check("rule doc tiles", verify_tiling(rspans2, RULE), [])
|
||
check(
|
||
"(b) '---' is not taggable",
|
||
any(s["kind"] in TAGGABLE and RULE[s["start"]:s["end"]].strip() == "---"
|
||
for s in rspans2),
|
||
False,
|
||
)
|
||
|
||
QUOTED = 'The alternative — collapse to "does it change what X asserts?" alone — is less safe.\n'
|
||
qspans = split_spans(QUOTED)
|
||
check("quoted-question doc tiles", verify_tiling(qspans, QUOTED), [])
|
||
check(
|
||
"(c) '?' inside a quotation does not create a fragment",
|
||
len([s for s in qspans if s["kind"] == "prose"]),
|
||
1,
|
||
f"got {[QUOTED[s['start']:s['end']] for s in qspans if s['kind'] == 'prose']}",
|
||
)
|
||
TWO = 'Is it sound? It is not.\n'
|
||
check(
|
||
"(c) a real sentence boundary still splits",
|
||
len([s for s in split_spans(TWO) if s["kind"] == "prose"]),
|
||
2,
|
||
"over-suppression would hide real boundaries",
|
||
)
|
||
|
||
print("\nDefects found by contact with a real package (splitter v1.2.0):")
|
||
|
||
ENUM = "**1. The canonical inquiry — a source, March 2026:**\n"
|
||
ENUM = "**1. The canonical inquiry, March 2026.** Then a second sentence.\n"
|
||
espans = split_spans(ENUM)
|
||
check("enumerator doc tiles", verify_tiling(espans, ENUM), [])
|
||
eprose = [ENUM[s["start"]:s["end"]] for s in espans if s["kind"] == "prose"]
|
||
check(
|
||
"(d) '**1.' does not orphan as a fragment",
|
||
any(t.strip().startswith("**1. The canonical") for t in eprose),
|
||
True,
|
||
f"got {eprose}",
|
||
)
|
||
YEAR = "The clause was added in 2026. The next sentence follows.\n"
|
||
check(
|
||
"(d) a real boundary after a year still splits",
|
||
len([s for s in split_spans(YEAR) if s["kind"] == "prose"]),
|
||
2,
|
||
"enumerator rule must not swallow '…in 2026. The next…'",
|
||
)
|
||
|
||
FM = '---\ntitle: "A title"\nstatus: "DRAFT. Nothing applied."\n---\n\nBody prose here.\n'
|
||
fmspans = split_spans(FM)
|
||
check("frontmatter doc tiles", verify_tiling(fmspans, FM), [])
|
||
check(
|
||
"(e) frontmatter is line-oriented, not shredded",
|
||
[FM[s["start"]:s["end"]] for s in fmspans if s["kind"] == "frontmatter"],
|
||
['title: "A title"\n', 'status: "DRAFT. Nothing applied."\n'],
|
||
)
|
||
check(
|
||
"(e) frontmatter stays taggable",
|
||
all(s["kind"] in TAGGABLE for s in fmspans if s["kind"] == "frontmatter"),
|
||
True,
|
||
"excluding it would shrink the quarantine in the author's favour",
|
||
)
|
||
|
||
print("\nForbidden-heading detector (§3.3) — must fire, and must not over-fire:")
|
||
for h in ("## Limitations", "## What this does not do", "### Open questions",
|
||
"## Caveats and scope", "## Assumptions",
|
||
# The FALSE PASS this screen actually gave, on a real package.
|
||
"## Part VII — Disconfirming evidence, which the steward asked to be carried",
|
||
"## Evidence against", "## Known gaps"):
|
||
check(f"fires on {h!r}", bool(FORBIDDEN_HEADING_RE.search(h)), True)
|
||
for h in ("## Part I — Grounding", "## The ruling", "## Conditions",
|
||
"## What changed", "## Part IV — Consequence-trace"):
|
||
check(f"quiet on {h!r}", bool(FORBIDDEN_HEADING_RE.search(h)), False)
|
||
check(
|
||
"screen is defeated by a bare section number — it is a screen, not a decision",
|
||
bool(FORBIDDEN_HEADING_RE.search("## Part VII")),
|
||
False,
|
||
"recorded so the pass is never read as a §2a verdict",
|
||
)
|
||
|
||
print("\nQ-resolution (§3.2) — must find a real quote and REJECT a fabricated one:")
|
||
from reduce import AXIOM_SOURCES, check_q_resolution, normalise_quote # noqa: E402
|
||
|
||
CLAUDE = AXIOM_SOURCES["CLAUDE.md"]
|
||
if CLAUDE.is_file():
|
||
# One genuine verbatim clause, one plausible fabrication.
|
||
QDOC = (
|
||
"> The loop is load-bearing\n"
|
||
"\n"
|
||
"> The loop is entirely optional and may be removed\n"
|
||
)
|
||
qspans2 = split_spans(QDOC)
|
||
qidx = [i for i, s in enumerate(qspans2) if s["kind"] in TAGGABLE]
|
||
check("q fixture tiles", verify_tiling(qspans2, QDOC), [])
|
||
check("q fixture has two quotable units", len(qidx), 2)
|
||
tags = {qidx[0]: ("Q", "CLAUDE.md"), qidx[1]: ("Q", "CLAUDE.md")}
|
||
probs = check_q_resolution(qspans2, QDOC, tags, {"CLAUDE.md": CLAUDE})
|
||
check("genuine quote resolves", any(f"span {qidx[0]}" in p for p in probs), False)
|
||
check(
|
||
"FABRICATED quote rejected",
|
||
any(f"span {qidx[1]}" in p for p in probs),
|
||
True,
|
||
"a §3.2 that cannot reject an invented quote checks nothing",
|
||
)
|
||
bad = check_q_resolution(
|
||
qspans2, QDOC, {qidx[0]: ("Q", "not-an-axiom-source.md")}, {"CLAUDE.md": CLAUDE}
|
||
)
|
||
check("unknown source rejected", len(bad), 1)
|
||
else:
|
||
failures.append("CLAUDE.md unresolvable — §3.2 control did not run")
|
||
print(" FAIL CLAUDE.md not found")
|
||
|
||
check(
|
||
"normalisation tolerates emphasis and rewrap, not word changes",
|
||
(normalise_quote("> **The loop** is\nload-bearing") == "The loop is load-bearing",
|
||
normalise_quote("The loop is load bearing") == "The loop is load-bearing"),
|
||
(True, False),
|
||
)
|
||
|
||
print("\nReal document — the splitter must tile actual governance prose:")
|
||
REAL = Path(__file__).resolve().parent.parent / "skill-harvest-fix-lane-JURIST-RULING-2026-08-01.md"
|
||
if REAL.is_file():
|
||
text = REAL.read_text(encoding="utf-8")
|
||
rspans = split_spans(text)
|
||
check(f"{REAL.name} tiles cleanly", verify_tiling(rspans, text), [])
|
||
else:
|
||
failures.append(f"real document absent: {REAL}")
|
||
print(f" FAIL {REAL.name} not found")
|
||
|
||
if failures:
|
||
print(f"\nINSTRUMENT NOT VERIFIED — {len(failures)} failure(s):")
|
||
for f in failures:
|
||
print(f" - {f}")
|
||
sys.exit(1)
|
||
|
||
print("\nAll gates verified, each shown failing on a fixture built to break it.")
|