[FIX] Reduction 01: a jurist ruling reduces to 8.5% under Kernel v1.0
First run of the reduction arm. Result: 4 of 47 assertive units survive. D=0, Q=0, A=0 — in a real jurist ruling not one unit is demonstrated-in-document and not one is a verbatim quote from a declared axiom source. Census: PERFORMATIVE 12, BLEND 9, UNSOURCED-FACT 8, TESTIMONY 6, INHERITED 4, UNSOURCED-QUOTE 3, PARAPHRASE 1. §6.1 asked whether a heavy quarantine means the kernel is too strict or our prose is full of unmarked assumptions. The census says neither: PERFORMATIVE and TESTIMONY are 42% of quarantines and are categories the kernel has NO TAG FOR. 'Design gate PASSED' is not an undemonstrated claim, it is a determination true by being uttered; 'I read CLAUDE.md in full' is testimony. A ruling that neither performed nor testified would not be a ruling. So the finding is a GENRE BOUNDARY — v1.0 models argumentative prose, a ruling is authoritative prose — and that boundary is nowhere stated in the kernel. Three gaps, one genre-independent: TESTIMONY, PERFORMATIVE, and PARAPHRASE. PARAPHRASE is the one that matters — Q demands verbatim, and any document reasoning from sources in its own words is untypeable. Plus a fourth, structural: the §1 axiom set is too narrow to reduce anything real (12 of 43 quarantines are UNSOURCED-* or PARAPHRASE). Deepest finding: §2c is satisfiable BY CONSTRUCTION but not BY REDUCTION. Splitting a blend means rewriting someone else's sentence, which is where translator bias lives. At 91.5% that is not reduction, it is authoring a new document with the original as a prompt — so on this genre the reduction arm COLLAPSES INTO the synthetic arm, inheriting its confirmation bias without its convenience. The two arms were adopted because they fail differently; that is the property at risk. n=1 and stated as such. The package genre splits to 109 taggable units and is NOT tagged. Falsifiable prediction recorded before the census: its Part I is 'Grounding (quoted verbatim)' and quotes CLAUDE.md directly, so Q should be non-zero there where it was zero here. Tooling: reduce.py + test_reduce.py, every gate shown FAILING on a fixture built to break it. The splitter shipped with three defects, all found by contact with a real document and none by review — third instance in three days: a '##' inside a fence kinded as a heading, '---' rules taggable, and a '?' inside a quotation splitting a sentence into a FRAGMENT. Fixed at v1.1.0 with regression controls; the third fix's own risk (lower-case suppression) is recorded and controlled.
This commit is contained in:
David F Glidden
Executable
+174
@@ -0,0 +1,174 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Positive controls for the reduction tooling.
|
||||
|
||||
Control Kernel v1.0 §3: "Each check ships with a positive control — a fixture it
|
||||
is shown to fail on — before any result from it is believed. An absence is not
|
||||
evidence until the instrument is shown capable of detecting presence."
|
||||
|
||||
So every gate below is shown FAILING on a fixture built to break it, and passing
|
||||
on one built not to. A gate only ever demonstrated passing has demonstrated
|
||||
nothing.
|
||||
|
||||
Usage: ./test_reduce.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from reduce import ( # noqa: E402
|
||||
FORBIDDEN_HEADING_RE,
|
||||
TAGGABLE,
|
||||
split_spans,
|
||||
verify_tiling,
|
||||
)
|
||||
|
||||
failures: list[str] = []
|
||||
|
||||
|
||||
def check(name: str, got, want, detail: str = "") -> None:
|
||||
if got != want:
|
||||
failures.append(f"{name}: expected {want!r}, got {got!r}. {detail}")
|
||||
print(f" FAIL {name}")
|
||||
else:
|
||||
print(f" ok {name}")
|
||||
|
||||
|
||||
SAMPLE = """# A heading
|
||||
|
||||
Some prose here. It has two sentences.
|
||||
|
||||
- a list item
|
||||
- another
|
||||
|
||||
> a quoted block
|
||||
|
||||
```
|
||||
code that must not be split. really.
|
||||
```
|
||||
|
||||
Final paragraph, e.g. with an abbreviation inside it. And a second sentence.
|
||||
"""
|
||||
|
||||
print("Tiling invariant — the gate everything else depends on:")
|
||||
spans = split_spans(SAMPLE)
|
||||
check("sample tiles cleanly", verify_tiling(spans, SAMPLE), [])
|
||||
check(
|
||||
"spans reproduce source byte-for-byte",
|
||||
"".join(SAMPLE[s["start"]:s["end"]] for s in spans),
|
||||
SAMPLE,
|
||||
)
|
||||
|
||||
print("\nPositive control — the gate must DETECT a broken tiling:")
|
||||
gap = [dict(s) for s in spans]
|
||||
gap[2]["start"] += 1 # open a one-byte hole
|
||||
check("gap detected", len(verify_tiling(gap, SAMPLE)) > 0, True, "gate blind to a gap")
|
||||
|
||||
overlap = [dict(s) for s in spans]
|
||||
overlap[2]["start"] -= 1 # overlap the previous span
|
||||
check("overlap detected", len(verify_tiling(overlap, SAMPLE)) > 0, True)
|
||||
|
||||
truncated = [dict(s) for s in spans[:-1]]
|
||||
check("truncation detected", len(verify_tiling(truncated, SAMPLE)) > 0, True)
|
||||
|
||||
print("\nSplitter behaviour:")
|
||||
prose = [s for s in spans if s["kind"] == "prose"]
|
||||
texts = [SAMPLE[s["start"]:s["end"]] for s in prose]
|
||||
check("abbreviation did not split 'e.g.'", sum("e.g." in t for t in texts), 1)
|
||||
check(
|
||||
"'e.g.' sentence not broken after the abbreviation",
|
||||
any(t.strip().startswith("Final paragraph, e.g. with") for t in texts),
|
||||
True,
|
||||
f"prose units: {texts}",
|
||||
)
|
||||
check("two sentences found in para 1", sum("Some prose here." in t for t in texts), 1)
|
||||
check(
|
||||
"code fence never becomes prose",
|
||||
any("code that must not be split" in SAMPLE[s["start"]:s["end"]] and s["kind"] == "code"
|
||||
for s in spans),
|
||||
True,
|
||||
)
|
||||
check(
|
||||
"list items are not prose",
|
||||
all("a list item" not in t for t in texts),
|
||||
True,
|
||||
)
|
||||
check("headings are taggable", "heading" in TAGGABLE, True,
|
||||
"kernel §4 rules a heading can assert")
|
||||
|
||||
print("\nDefects found by contact with a real ruling (splitter v1.1.0):")
|
||||
|
||||
FENCED = """Ready to paste:
|
||||
|
||||
```
|
||||
## REVIEWED-85 — a heading INSIDE a fence
|
||||
**Date:** 2026-08-01
|
||||
```
|
||||
|
||||
After the fence.
|
||||
"""
|
||||
fspans = split_spans(FENCED)
|
||||
check("fenced doc tiles", verify_tiling(fspans, FENCED), [])
|
||||
check(
|
||||
"(a) '##' inside a fence is code, not a taggable heading",
|
||||
any(s["kind"] == "heading" and "REVIEWED-85" in FENCED[s["start"]:s["end"]]
|
||||
for s in fspans),
|
||||
False,
|
||||
"quoted content must not become structure of the quoting document",
|
||||
)
|
||||
|
||||
RULE = "Some prose.\n\n---\n\nMore prose.\n"
|
||||
rspans2 = split_spans(RULE)
|
||||
check("rule doc tiles", verify_tiling(rspans2, RULE), [])
|
||||
check(
|
||||
"(b) '---' is not taggable",
|
||||
any(s["kind"] in TAGGABLE and RULE[s["start"]:s["end"]].strip() == "---"
|
||||
for s in rspans2),
|
||||
False,
|
||||
)
|
||||
|
||||
QUOTED = 'The alternative — collapse to "does it change what X asserts?" alone — is less safe.\n'
|
||||
qspans = split_spans(QUOTED)
|
||||
check("quoted-question doc tiles", verify_tiling(qspans, QUOTED), [])
|
||||
check(
|
||||
"(c) '?' inside a quotation does not create a fragment",
|
||||
len([s for s in qspans if s["kind"] == "prose"]),
|
||||
1,
|
||||
f"got {[QUOTED[s['start']:s['end']] for s in qspans if s['kind'] == 'prose']}",
|
||||
)
|
||||
TWO = 'Is it sound? It is not.\n'
|
||||
check(
|
||||
"(c) a real sentence boundary still splits",
|
||||
len([s for s in split_spans(TWO) if s["kind"] == "prose"]),
|
||||
2,
|
||||
"over-suppression would hide real boundaries",
|
||||
)
|
||||
|
||||
print("\nForbidden-heading detector (§3.3) — must fire, and must not over-fire:")
|
||||
for h in ("## Limitations", "## What this does not do", "### Open questions",
|
||||
"## Caveats and scope", "## Assumptions"):
|
||||
check(f"fires on {h!r}", bool(FORBIDDEN_HEADING_RE.search(h)), True)
|
||||
for h in ("## Part I — Grounding", "## The ruling", "## Conditions",
|
||||
"## What changed"):
|
||||
check(f"quiet on {h!r}", bool(FORBIDDEN_HEADING_RE.search(h)), False)
|
||||
|
||||
print("\nReal document — the splitter must tile actual governance prose:")
|
||||
REAL = Path(__file__).resolve().parent.parent / "skill-harvest-fix-lane-JURIST-RULING-2026-08-01.md"
|
||||
if REAL.is_file():
|
||||
text = REAL.read_text(encoding="utf-8")
|
||||
rspans = split_spans(text)
|
||||
check(f"{REAL.name} tiles cleanly", verify_tiling(rspans, text), [])
|
||||
else:
|
||||
failures.append(f"real document absent: {REAL}")
|
||||
print(f" FAIL {REAL.name} not found")
|
||||
|
||||
if failures:
|
||||
print(f"\nINSTRUMENT NOT VERIFIED — {len(failures)} failure(s):")
|
||||
for f in failures:
|
||||
print(f" - {f}")
|
||||
sys.exit(1)
|
||||
|
||||
print("\nAll gates verified, each shown failing on a fixture built to break it.")
|
||||
Reference in New Issue
Block a user