David/dotfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[FIX] PENDING-151 step 1: the formation diff, run — with the confound that would have misled step 2

Browse Source 
Mechanical, reproducible, no judgement. The instrument emits counts and word lists and
stops, and its controls assert structurally that it renders no verdict: no 'substantive',
no 'stylistic', no 'register' column exists to fill in. The routing is built into the tool
because the executor is ONE OF THE TWO FORMATIONS BEING COMPARED, and PENDING-151 says
outright that no disclosure repairs that, only routing does.

⚠ THE GATE WAS NOT MET AND THE ITEM SAYS SO. Its own pre-registration required a jurist or
steward commitment to step 2 BEFORE step 1 ran. The steward authorized the work; nobody has
committed to step 2. An unjudged diff table invites the nearest available reader to judge
it, and that reader is the barred party. Recorded so the table's inertness is visible.

⚠ AND STEP 1 FOUND A CONFOUND IN ITS OWN PRE-REGISTERED MEASURE. The Claude arm is longer
in 9 of 9 pairs, 1.41x-3.40x. 'Terms present in one arm and absent from the other' rises
with length by construction, so the raw counts measure length at least as much as
formation. Length-normalised columns added — and declared imperfect, because whether a term
counts as absent depends on the OTHER arm's length too. Both columns remain
length-sensitive in opposite directions. A length-matched instrument would be clean and is
not built.

⚠ PROPOSITIONS NOT EXTRACTED, declared as a limit rather than silently dropped: extraction
requires reading for claims, and the only reader at step 1 is the party barred from step 2.

Census re-run rather than inherited: 19,479 words EXACT, 9 pairs, 6 sessions, 3 protocols
all confirmed. File counts drift 1-3 on AppleDouble churn, which is why '55 files' was
never stable.

⚠ Fifth self-referential control bug of the day, in a script that does not import the
helper built for it. Needle assembled. The rule, now plain: a control reading a corpus that
contains the control must BUILD its needle, never write it.

Archive untouched; filename defects preserved as the 2025 record.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J6hZXNYSxEfZseBGTni4sf
This commit is contained in:
David F Glidden
2026-08-25 18:53:04 +02:00
co-authored by Claude Opus 5
parent cb8b50764c
commit 0f95c873f5
3 changed files with 471 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/chamber-v1-formation-diff.py
Executable
+199
View File
@@ -0,0 +1,199 @@
#!/usr/bin/env python3
"""PENDING-151 step 1 — the v1 Chamber archive, formation diff. MECHANICAL ONLY.
"Step 1 — mechanical, executor, reproducible. Per pair, extract distinctive
content: terms, named entities, and propositions present in one arm and absent
from the other. Output is a diff table, NOT A JUDGEMENT. Checkable by re-running."
⚠ THIS SCRIPT PRODUCES NO VERDICT AND MUST NOT BE MADE TO. Step 2 — "does this
divergence carry different CONTENT, or the same content in a different REGISTER?" — is
reserved to the jurist or the steward, and PENDING-151 says outright: "This is the step
the finding rests on and the executor may not take it."
⚠ AND THE REASON IS NOT PROCEDURAL. The executor IS ONE OF THE TWO FORMATIONS BEING
COMPARED. It would be judging whether its own formation's divergence from GPT is
substantive, on a question bearing on whether its own seat is a real check. PENDING-151:
"No disclosure repairs that; only routing does." So the routing is built into the tool:
it emits counts and word lists and stops.
⚠ PROPOSITIONS ARE NOT EXTRACTED, and this is a declared limit rather than an omission.
The pre-registration names three levels — terms, named entities, propositions. The first
two are mechanical. Proposition extraction is not: it requires reading for claims, which
is interpretation, and the only interpreter available here is the party barred from
step 2. Manufacturing a "propositions" column with a model would be step 2 wearing step
1's clothes. It is therefore left to the step-2 reader, who is reading the pairs anyway.
Pre-registered before any file was opened; see PENDING-151. Re-run to check.
"""
import os
import re
import sys
import collections
ROOT = os.path.expanduser(
"~/_Dev/animal-davidglidden-eu/chamber-sessions-private/2025")
MIN_FREQ = 2 # a term must appear twice in one arm to count as distinctive
TOP_N = 25 # per arm, per pair, in the table
STOP = set("""a about above after again against all am an and any are aren as at be because been
before being below between both but by can cannot could couldn did didn do does doesn doing don down
during each few for from further had hadn has hasn have haven having he her here hers herself him
himself his how i if in into is isn it its itself just me more most my myself no nor not now of off
on once only or other ought our ours ourselves out over own same shan she should shouldn so some such
than that the their theirs them themselves then there these they this those through to too under
until up very was wasn we were weren what when where which while who whom why will with won would
wouldn you your yours yourself yourselves s t don ll re ve m one also may might must shall upon
""".split())
def tokens(text):
return [w for w in re.findall(r"[A-Za-z][A-Za-z'-]+", text)]
def content_terms(text):
return collections.Counter(w.lower() for w in tokens(text)
if w.lower() not in STOP and len(w) > 3)
def entities(text):
"""Capitalised runs not at sentence start. Crude, deterministic, re-runnable."""
out = collections.Counter()
for sent in re.split(r"(?<=[.!?])\s+|\n\n", text):
ws = re.findall(r"[A-Z][A-Za-z'-]+(?:\s+[A-Z][A-Za-z'-]+)*", sent[1:] if sent else "")
for e in ws:
if e.lower() not in STOP and len(e) > 3:
out[e] += 1
return out
def pairs():
real = []
for dp, _, fn in os.walk(ROOT):
for f in fn:
if f.startswith("._") or f == ".DS_Store":
continue
real.append(os.path.join(dp, f))
found = collections.defaultdict(dict)
for f in real:
b = os.path.basename(f).strip() # ⚠ leading-space defect, NOT repaired
m = re.match(r"\[(\w[\w-]*)\](gpt|claude)-raw", b)
if m:
proto, arm = m.group(1), m.group(2)
else:
m2 = re.match(r"(gpt|claude)-raw", b)
if not m2:
continue
proto, arm = "standard", m2.group(1)
sess = os.path.relpath(os.path.dirname(f), ROOT).split(os.sep)[0]
found[(sess, proto)][arm] = f
return {k: v for k, v in sorted(found.items()) if "gpt" in v and "claude" in v}
def main():
ps = pairs()
print("# PENDING-151 step 1 — v1 Chamber formation diff (MECHANICAL, NO JUDGEMENT)\n")
print(f"Pairs: **{len(ps)}**. Generated by `scripts/chamber-v1-formation-diff.py`; "
f"re-run to check. ⚠ **No column here says whether a divergence is substantive "
f"or stylistic. That is step 2 and the executor may not take it.**\n")
rows = []
for (sess, proto), arms in ps.items():
g = open(arms["gpt"], encoding="utf-8", errors="replace").read()
c = open(arms["claude"], encoding="utf-8", errors="replace").read()
gt, ct = content_terms(g), content_terms(c)
ge, ce = entities(g), entities(c)
g_only = {w: n for w, n in gt.items() if n >= MIN_FREQ and w not in ct}
c_only = {w: n for w, n in ct.items() if n >= MIN_FREQ and w not in gt}
ge_only = {e: n for e, n in ge.items() if e not in ce}
ce_only = {e: n for e, n in ce.items() if e not in ge}
shared = set(gt) & set(ct)
union = set(gt) | set(ct)
rows.append((sess, proto, len(g.split()), len(c.split()),
len(g_only), len(c_only), len(ge_only), len(ce_only),
len(shared) / len(union) if union else 0))
print(f"\n## {sess} · `{proto}`\n")
print(f"| | GPT arm | Claude arm |")
print(f"|---|---|---|")
print(f"| words | {len(g.split()):,} | {len(c.split()):,} |")
print(f"| distinct content terms | {len(gt):,} | {len(ct):,} |")
print(f"| **terms ≥{MIN_FREQ}× in this arm, absent from the other** | "
f"**{len(g_only)}** | **{len(c_only)}** |")
print(f"| named entities absent from the other | {len(ge_only)} | {len(ce_only)} |")
print(f"| shared-term Jaccard | colspan | {len(shared)/len(union):.3f} |")
for label, d in (("GPT-only terms", g_only), ("Claude-only terms", c_only)):
top = sorted(d.items(), key=lambda kv: -kv[1])[:TOP_N]
print(f"\n**{label}** ({len(d)}): " +
(", ".join(f"{w}·{n}" for w, n in top) or "—"))
for label, d in (("GPT-only entities", ge_only), ("Claude-only entities", ce_only)):
top = sorted(d.items(), key=lambda kv: -kv[1])[:TOP_N]
print(f"\n**{label}** ({len(d)}): " +
(", ".join(f"{e}·{n}" for e, n in top) or "—"))
print("\n---\n\n## Summary — counts only\n")
print("| session | protocol | GPT w | Cl w | len ratio | GPT-only | Cl-only | "
"GPT-only /1k | Cl-only /1k | Jaccard |")
print("|---|---|---|---|---|---|---|---|---|---|")
for r in rows:
ratio = r[3] / r[2] if r[2] else 0
gk = 1000 * r[4] / r[2] if r[2] else 0
ck_ = 1000 * r[5] / r[3] if r[3] else 0
print(f"| {r[0]} | {r[1]} | {r[2]:,} | {r[3]:,} | {ratio:.2f}× | {r[4]} | {r[5]} "
f"| {gk:.1f} | {ck_:.1f} | {r[8]:.3f} |")
ratios = [r[3] / r[2] for r in rows if r[2]]
print(f"\n⚠ **THE DOMINANT STRUCTURAL FEATURE IS LENGTH, AND IT CONFOUNDS THE RAW "
f"COUNTS.** The Claude arm is longer in **{sum(1 for x in ratios if x > 1)} of "
f"{len(ratios)} pairs**, ratio {min(ratios):.2f}×–{max(ratios):.2f}× "
f"(median {sorted(ratios)[len(ratios)//2]:.2f}×). A longer text yields more "
f"terms-absent-from-the-other BY CONSTRUCTION, so the bolded raw counts above "
f"measure length at least as much as formation. The `/1k` columns divide each "
f"arm's distinctive-term count by its own length and are the columns to compare. "
f"Reported this way because a step-2 reader handed the raw counts alone would be "
f"reading a length artifact as a formation difference — and would be right to, "
f"since nothing in the table said otherwise.")
print(f"\n⚠ **This is a mechanical observation about the corpus, not a finding about "
f"the arms.** Why one arm is longer — formation, prompt, protocol, or the 2025 "
f"settings of either model — is not answerable from these files and is not "
f"claimed here.")
j = [r[8] for r in rows]
print(f"\nJaccard over {len(j)} pairs: min {min(j):.3f}, median "
f"{sorted(j)[len(j)//2]:.3f}, max {max(j):.3f}")
print("\n⚠ **A Jaccard is a lexical overlap, not a content measure.** Two arms saying "
"the same thing in different words score low; two arms saying opposite things "
"in the same vocabulary score high. It is reported because it is reproducible, "
"and it decides nothing.")
def selftest():
checks, failed = [], []
def ck(n, c):
checks.append(n)
(failed.append(n) if not c else None)
src = open(__file__, encoding="utf-8").read()
body = src[src.index("def main()"):src.index("def selftest")]
# V1 — the tool cannot render a verdict, structurally.
for word in ("substantive", "stylistic", "register", "verdict", "judge"):
ck(f"V1 emits no '{word}' column", f'"{word}' not in body.lower())
ck("V1n the predicate can fail", '"words' in body or "words" in body)
# V2 — propositions are declared absent, not silently skipped.
ck("V2 propositions declared as a limit", "PROPOSITIONS ARE NOT EXTRACTED" in src)
# V3 — extraction is deterministic.
t = "The Owl and the Emblem. Alpha beta gamma alpha beta alpha."
ck("V3 terms deterministic", content_terms(t) == content_terms(t))
ck("V3 entities skip sentence-initial", "The" not in entities(t))
ck("V3n entities catch mid-sentence caps", any("Emblem" in e for e in entities(t)))
# V4 — pairing tolerates the archive's filename defects without repairing them.
ck("V4 filename defects tolerated by strip()", ".strip()" in src)
# ⚠ NEEDLE ASSEMBLED. Fifth time today a control was written with a literal needle
# and matched itself. The tarbuckle module has source_lacks() for this; importing it
# here would couple a chamber instrument to the fool's, so the idiom is inlined
# instead. The rule, now stated plainly: a control reading a corpus that contains the
# control must BUILD its needle, never write it.
_no_rename = "os." + "rename"
ck("V4n defects are not renamed", _no_rename not in src)
ck("V4nn the assembled predicate can fail", ("os." + "walk") in src)
for n in checks:
print(f" {'FAIL' if n in failed else 'ok '} {n}")
print(f"{len(checks)-len(failed)}/{len(checks)} controls passed")
return 1 if failed else 0
if __name__ == "__main__":
sys.exit(selftest() if "--selftest" in sys.argv else (main() or 0))