Files

T

David F Glidden 62fa6d857e session 2026-07-04: Chamber spec v2.0 ratified+shipped (a10d6bf); new session memory + ledger + skill-harvest (/spec-amendment build-ready, wrap-up §4.a patch)

2026-07-04 11:13:20 +02:00

32 KiB

Raw Permalink Blame History

name, description, metadata

name

description

metadata

session-ledger-2026-07-03

Practice-of-return ledger maintained by /symmetria — returns, open horizons, recalibrations, authorization moves, sub-agent dialogues, bypasses.

node_type	type	originSessionId
memory	feedback	5b7ce298-123b-447a-9b86-2442c30bec1f

Session Ledger — 2026-07-03

Returns

2026-07-03 — check before writing audit_corpus.py. Three commitments logged: (1) the verify_graduation.py refactor touches a GOVERNED gate (PENDING-44) — must be behavior-preserving + PROVEN byte-identical stdout on a sample before trusting; subprocess fallback if proof fails. (2) Scope: conventions+apparatus on the 332 non-Loeb only (Loeb 952 = separate DSL tier; the convention gate would false-FAIL them); text-health on all 1284. (3) Apparatus check CLASSIFIES + emits evidence counts (none/paired-ok/orphaned/stranded-suspect), never adjudicates; validate the classifier against Sennett (→paired-ok) + a real editorial-note file (→not-stranded) before trusting — per census-through-a-pattern flag. Data already surfaced: 8 files with live [^]: machinery vs 68 with a Notes heading ⇒ ~60 stranded-suspects.
2026-07-03 — apparatus classifier blind-spot CAUGHT by validating against real files (not just synthetic self-test): Taruskin/Beethoven had 89/131 [^x] refs with ZERO defs (all dangling) yet were classified notes-editorial (benign) because an editorial-style heading matched and the orphaned branch only fired when defs>0. Fix: refs-present-without-defs → orphaned, caught before the heading fallthrough. Without the real-file validation the map would have marked ~dozens of damaged files clean. The census-through-a-pattern flag paid off exactly as predicted.
2026-07-03 — GOVERNED-gate touch (PENDING-44): refactored verify_graduation.py to expose collect_checks()/resolve_spec_path() so the auditor and the gate share one convention source. Proven byte-identical stdout + exit on PASS(Sennett) and FAIL(Plato) samples vs git HEAD before trusting. Behavior-preserving [FIX]; paper note owed alongside PENDING-44/45.

Open horizons

2026-07-03T10:50 — CONTEXT CLEARED + re-woke (9 min after wrap). Thread CONFIRMED unchanged: Phase 2 — draft chamber-library-specification.md v2.0 (semver-MAJOR, superseding) + generative validator. FIRST ACT non-negotiable: read the CURRENT spec end-to-end BEFORE drafting (the day's source-not-derived lesson turned on the spec itself — I worked from audits all day, never the spec text). Then §III the CTS addressing spine. Standing return: do NOT rush; dwell-work on the constitution; do NOT draft against my model of the spec. chamber-library main ahead 3 unpushed; _scratch/+Alexander untracked = pre-existing, not mine.
2026-07-03 — MAP BUILT. audit_corpus.py + corpus-quality-ledger.tsv (1284 rows) + interpreted corpus-quality-map-2026-07-03.md with proposed waves. Distribution: Loeb 930/952 clean (corpus healthier in aggregate than the 332-view); non-Loeb 8 clean, 324 A-convention (mechanical), 110 B-textdamage, 70 C-apparatus (NEW axis). Wave A (frontmatter migration, ~99% mechanical, clears 170 A-only to green) recommended as first countdown. AWAITING steward authorization to run a wave — do not begin unilaterally.
2026-07-03T00:00 — Woke into the confirmed pulling thread: build scripts/audit_corpus.py + apparatus check → full quality ledger (_curation/corpus-quality-ledger.tsv) of ~1,284 works. Verified the map does NOT yet exist. Standing return to hold: do not resume graduating reactively — the mole-by-mole reflex is the pattern that produced last session's distress; measure the whole first.
2026-07-03 — steward challenged the framing (correctly): are the TOOLS spec-compliant before we run any wave? "The tool IS the wave — a stale cleaner run at scale re-corrupts 300 files." Confirmed gaps from reading spec+runbook+tool-log: (1) §VII gate-fix is [OPEN] — verify_conversion heading_density FALSE-FAILS legitimately sectionless works (La Chute), so the map's 59 "B2 structure-recovery" count is CONTAMINATED (conflates damage with sectionless-by-nature). (2) NO gate checks apparatus/footnote-pairing (§V keep_notes) — why 70 damaged files graduated. (3) NO per-language dict-validity check — passed Levi's capacitv/Alanichaean char-substitution. (4) 13/15 mutating/gate tools are STANDALONE (don't reference the governed spec → nothing prevents drift). (5) graduate_to_canonical missing the 2026-07-02 source-archiving wiring (TODO). ⇒ launching fleet compliance audit (3 agents) before authorizing Wave A.
2026-07-03 — MAP CORRECTION (verified): of 59 heading_density fails, 46 are the §VII sectionless FALSE-FAIL (spec-valid — Plato Republic, Sontag, Berger's About Looking w/25 breaks) and only 13 are genuine structure-loss (some of those, e.g. Rilke Malte, may also be sectionless). B2 was inflated ~4.5×. Running B2 blind would have FORCED headings onto ~46 sectionless works = §II corruption. The gate needs the §VII fix before B2 is even definable. Steward's tools-first instinct vindicated.
2026-07-03 — RETIRE candidates (steward asked: complete kit, nothing useless). Full scripts/ census by ref-count: ref=0 → clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings; + clean_pdf_running_heads (hardcoded "Auerbach Mimesis" — book-specific); + clean_ocrmac_pagination (superseded by normalize_ocr per runbook). Candidates only — agents confirming redundant-vs-unique before retiring (don't assert-absence-from-grep).

v2.0 read findings (2026-07-03T11:00 — first full end-to-end read of chamber-library-specification.md v1, against the day's doctrine; jurist-ruled)

The source-not-derived discipline at the highest-stakes point: every prior ruling was built on audits/briefs ABOUT the spec, never the spec text. First full read caught a real one.

F1 (LOAD-BEARING) — §III page-number exclusion vs. the canonical anchors just ratified. §III (L80–82) blanket-excludes page numbers as "the wrong unit, edition-specific." But Stephanus/Bekker/Diels-Kranz — the recovered anchors today's §III re-ruling cites the Loeb by — ARE page numbers. Would have shipped a flat self-contradiction. Jurist ruling: RECONCILED via the PORTABILITY TEST, not a shape/exception carve-out. Excluded iff resolving requires specifying a particular edition/printing ("p.41" needs "Summit 1988"); included iff it resolves to the same location across every circulating edition (Stephanus prints in every margin). Test on the property, not the surface form. Won't need re-amending per genre. THIS is the doctrine to draft §III from.
F2 — quote-primary → anchor-primary shift. CONFIRMED: anchor-primary where a recovered canonical anchor exists; verbatim-quote-primary only for the genuine-fragment/sectionless residual. Must be an EXPLICIT §III revision, not smuggled.
F3 — addressing substrate spans III+IV+V. CONFIRMED SHAPE, one caution: JOIN the addressing substrate (shared pointer scheme) across III/IV/V, but keep the functions distinct — citation-for-reader (III) / provenance-for-audit (IV) / apparatus-integrity (V). Don't collapse into one section.
F4 — §V demotes apparatus placement to "operational, a runbook decision" (L138–142). CONFIRMED SUPERSEDED: gap-2's ratified substrate-sharing means apparatus placement can't be merely operational. v2.0 §V explicitly supersedes.
F5 — §VII internal status contradiction (L194 [OPEN] vs L258 "conformant"). SOURCE-CHECKED: verify_conversion.py:36–44,112–125 IMPLEMENTS the sectionless exception (strips leading frontmatter, matches real markdown breaks only, still flags §II.2 literal-glyph-separator defect). Body [OPEN] is STALE → v2.0 marks §VII SETTLED.
F6 — stale figures + §XI framing. CONFIRMED: §VI "84 of ~1,285" + corpus-size → declared-data not hardcode (full DSL = ~2,041); amendment process EXTENDS §XI, no new governance organ.
Source-check (b): Stephanus/Bekker SOURCE-VERIFIED in loeb.dsl ([c dimgray]St. II p. 203[/c], 381a, 1441a), carried in the semantic layer extract_loeb_dsl.py:47 discards. Flattening claim holds.

v2.0 drafting progress (spine-first)

2026-07-03T11:15 — §III GATE-PASSED (jurist editor-gate). 4 items: (1) dormant structured-unanchored contingency = MOOT (test's generality closes it — intrinsic structure alone is admitted, no external-scheme requirement); (2) forward-note to §VII: gate checks a DECLARED LOCATOR REGISTRY, not live-derived portability ("registry is the gate, not the word Loeb") — captured in §VII placeholder; (3) header §II "carried unchanged" → CORRECTED to revised; (4) §VI [CARRIED]→[REVISED], Finding 6 NOT yet closed (declared-data fix owed) → added to queue. All 3 actionable applied to draft.
2026-07-03T11:20 — §II DRAFTED. Carried §II.1/.2/.4 from v1; two substantive additions: (a) break-normalization is CONSTITUTIONAL — literal glyph-run separators (• • •) = no represented structure = §II.2 defect, normalize to ---; distinguishes sectionless-by-nature from structure-flattened (verify_conversion enforces). (b) NEW §II.3 — admitted anchors represented as TYPED anchors + recoverable (per disambiguation map); carrier locators dropped off-canon. The anchor REPRESENTATION FORM (inline marker vs bound structural map) flagged [OPEN] — source-checked the §III re-ruling + disambiguation map: doctrine ratified, representation is "downstream engineering gated by validator," so NOT invented — named as bounded fork (constrained by clean-prose §II.1 + verbatim §V). This is the one place needing jurist eyes on §II.
2026-07-03T11:35 — §II GATE-PASSED (jurist). Two corrections applied: (1) §II.2 attestor gap — represented-break test is NECESSARY-NOT-SUFFICIENT (catches present-but-unrepresented, NOT removed-entirely-in-conversion = the twice-today failure); sectionless: true must be a POSITIVE ATTRIBUTED attestation (attestor + evidence), enforcement forward-noted to §VII. (2) §II.3 representation-form was NOT actually an open fork — jurist RULED inline typed marker REQUIRED (anchor = citation unit = content; standing invariant: grounding gate verifies only vs canonical text, never sidecar → sidecar-only anchor structurally can't support §III; + offsets drift). Open = exact syntax + optional derived perf-index. Better than my OPEN framing. +2 forward notes: §IV (anchor ≠ carrier-provenance), §V (word-guard excludes anchor syntax).
2026-07-03T11:45 — §V DRAFTED (verbatim guarantee & apparatus integrity). Carried 3-tier + minimize-catalogue from v1. NEW: (a) conversion record MANDATORY every path (W3C-PROV shape; no record → can't enter canon); (b) TWO gate-functions named + kept distinct — content-grounding (canonical-text-only, preserves standing invariant) vs process-integrity (reads the conversion record, the fn extended today); (c) Finding 4 apparatus supersession — apparatus ANCHOR is constitutional (anchor-to-location not renumberable-note-number, TEI ch.13), display FORM stays operational-but-uniform; (d) inline-anchor excluded from prose-word-guard (§II.3 realized). Next: §IV.
2026-07-03T11:55 — §V GATE-PASSED (jurist; apparatus split praised as sharper-than-ruled). One addition: word-guard excludes the anchor → so what verifies anchor CONTENT correctness? Fix (in-architecture): anchor recovery is a declared Tier-2 normalization → process-integrity verifies it vs source (not the word-guard). Clause added: "division of labour is total — word-guard proves prose unchanged, process-integrity proves recovered anchor faithful; no part unverified."
2026-07-03T12:05 — §IV DRAFTED (provenance axes & edition-identity). Carried 4 axes + graceful-degradation. REVISED source: v1 breadcrumb → EDITION-IDENTITY (the frame the §III anchor resolves under); two roles — classical works = CTS-URN edition-as-identity + portable anchor = full citation coordinate; modern works = source printing = provenance-only (page-nos fail portability). NEW: edition-identity (canon axis) ≠ carrier-page provenance (off-canon record) — forward-note realized. Finding-3 close: substrate joined across III/IV/V, §IV function = provenance-for-audit + temporal/genealogy, kept distinct. Next: §VI (short — declared-data fix).
2026-07-03T12:15 — §IV GATE-PASSED (jurist; the edition-identity move praised — CTS work/version+passage "the one structure that matches what Chamber needs to verify," since Stephanus locates but wording differs across editions). +1 forward note to §Tiering&Fence: tier-1 promotion criterion "source-identity confirmed" target IS §IV's edition-identity field — link, don't re-derive. Added.
2026-07-03T12:20 — §VI DRAFTED (short, as flagged). Architecture CARRIED from v1 (identity tag + reading-index sidecar hash-bound; engine-side voice manifest; the sole grounding invariant). Only REVISION = Finding 6: migration population is DECLARED DATA (read from catalogue.yaml census), not hardcoded — corpus in flux (~952→~2,041 full-DSL run); spec names where-the-count-lives, not the count (gap-4 discipline reaching §VI). Cross-linked the grounding invariant to §V content-grounding. Next: §VII.
2026-07-03T12:30 — §VI GATE-PASSED CLEAN (jurist; hash-binding praised as the §II.3 discipline showing up correctly elsewhere; stale-sidecar enforcement correctly out-of-scope since grounding gate never reads a reading-index). No conditions.
2026-07-03T12:45 — §VII DRAFTED (the dense one). Source-checked graduation-spec.yaml first (v1, PENDING-44: "SINGLE SOURCE OF TRUTH… operationalizes §§II–VI, spec wins on conflict"; gaps_BLOCK_graduation real) → drafted faithful to what EXISTS vs what v2.0 newly MANDATES. Sections: storage-tiers carried (+ orthogonal to §Tiering&Fence evidence-tiers); TWO-gate division (verify_conversion=quality / verify_graduation=convention-compliance driven by graduation-spec.yaml); declared-list-IS-the-gate = gap-7 generative principle PARTLY REALIZED, v2.0 completes (JSON-Schema + encode registry/attestation); honest-coverage (body_word_conservation tracked-unimplemented, visible not silent); both §V gate-fns at boundary; declared locator registry (§III mechanism — unregistered shape → human PROPOSAL decision, never silent); sectionless-attestation enforced (bare flag FAILS); gate-fix SETTLED source-verified; PASS-BUT-FALSELY sharpened (gate-itself-can-be-PBF, 06-28). Next: §IX (short-ish).
2026-07-03T13:00 — §VII GATE-REVIEW: 5 closures confirmed, 1 gap → RESOLVED. Jurist praised PASS-BUT-FALSELY's class-fix as §III's portability move independently re-derived (generative principle generalizing on its own). Gap: word-guard order-blindness (multiset can't catch reading-order scramble). Jurist honest-uncertain whether sequence-check survives every normalization. My resolution (diagnosed the class, didn't defer): TWO LAYERS — (1) cleaning raw→canonical: strengthen to ORDER-SENSITIVE (no permitted Tier-2 norm reorders body prose); the ONE false-positive case = apparatus display-form move (inline↔endnote) → resolved by PARTITION (prose order-sensitive, apparatus anchor-bound) which falls out of §V's already-ratified anchor-to-LOCATION doctrine; empirical pipeline-confirmation NAMED as a gated tool-evolution check. (2) conversion source→raw OCR column-scramble: word-guard STRUCTURALLY can't reach (both sides scrambled) → reading-order-coherence + eyeball = GENUINE ceiling there. Answered the jurist's ceiling-vs-default question per-layer. Refined §V word-guard clause (multiset→order-sensitive+anchor-bound). +storage≠evidence forward note to §Tiering&Fence. Awaiting jurist confirm of the resolution.
2026-07-03T13:20 — §IX DRAFTED (was OPEN → PARTIAL). census(catalogue.yaml)→coverage-ledger (what corpus can/can't answer, derived from census+fence); NEW evidence-tier column (gap-6 surfaces: coverage reported WITH confidence, converted≠fenced-in); NEW freshness invariant (stale coverage=silent failure; bound to census state, detectable-not-silent, honest-degrade). Silence-as-finding representation left OPEN (named not guessed).
2026-07-03T13:35 — §Tiering&Fence DRAFTED. Source-checked gap6-evidence-tier-data first. CRITICAL governance state: tiers RATIFIED but promotion criterion ESCALATED (own PENDING, no file promoted until ruled) — drafted tiers as settled, criterion as clearly-marked PROPOSED-shape. Evidence tiers V-DSL 952/V-TEXT 239/V-SCAN 50/V-SUSPECT 29/V-NONE 14 (counts as declared-data census snapshot). Fence tiers converted/verified/fenced-out. 93% deterministic path. Realized both forward notes (storage≠evidence; §IV edition-identity coupling). Governability = Seck 2604.10658.
2026-07-03T13:50 — §Governance&Amendment DRAFTED as [PROPOSED]. ⚠ CAUGHT MY OWN OVERCLAIM: header said "the RATIFIED amendment process" but the authoritative RULING doc says gap-7 + amendment-process corollary = RETURNED for joint jurist+steward co-ratification, NOT decided. My session-note said "ratified" — CONFLICT → trusted the ruling DOC (source-not-derived / verify-on-conflict), drafted whole section [PROPOSED], corrected header to "first exercise of… pending joint co-ratification." The recursion (v2.0 IS the process's first exercise, running provisionally) named honestly. Process = RFC-supersession + change-class("does this change what a gate accepts") + semver-MAJOR-re-verifies-consumers + migration-window(question-C: verified_under/re-verify-pending/N=2-provisional/FIX-extension). ⇒ ESCALATE-adjacent: the ratification state of gap-7 + the amendment process needs steward confirmation — this is the one place the draft's authority claim must be checked, and gap-7 was returned specifically TO the steward.
2026-07-03T13:55 — ALL 9 DRAFTING-QUEUE SECTIONS DONE. Remaining: assemble carried §0/§I/§X/§XI verbatim into the file; final header/status pass; whole-draft jurist read. Six sections gate-passed individually (III/II/V/IV/VI/VII-mostly); §IX/§Tiering&Fence/§Governance&Amendment await gate.
2026-07-03T14:05 — GAP-7 JOINTLY CO-RATIFIED (steward half + jurist half, both recorded). §Governance&Amendment [PROPOSED]→[RATIFIED] (all subsection tags + header + read-first note updated). Jurist's reason: better a process that survived a full day of real use than one that only survived design review. §IX GATE-PASSED (freshness invariant reuses detectable-staleness pattern, discipline-become-practice). §Tiering&Fence: jurist CAUGHT my stale-reference — promotion-criterion tag said "escalated/not-ratified" citing the FIRST ruling; the shape was ratified TWICE (consolidated + three-questions). Corrected → RATIFIED-SHAPE·OPERATION-BLOCKED (two operational blocks: (a) §VII order-blindness resolution, (b) edition-identity coupling); NOT awaiting a ruling. The irony noted in-text: stale-reference failure caught in the doc about preventing it. §VII order-blindness resolution effectively accepted-in-frame (jurist named "strengthened comparison" = what I drafted, as the operational-unblock path).
2026-07-03T14:10 — v2.0 SUBSTANCE COMPLETE. All 9 sections gate-passed/ratified/corrected. REMAINING = mechanical assembly only: carry §0/§I/§VIII/§X/§XI verbatim from v1, reorder to final sequence (§0→I→II→III→IV→V→VI→VII→VIII→IX→Tiering&Fence→X→XI→Governance&Amendment), final header/status pass, whole-draft read. Proposed ordering to steward before reorganizing.
2026-07-03T14:25 — v2.0 ASSEMBLED (self-contained, 744 lines). Steward approved section order → Wrote final: §0·I carried verbatim; §II–VII drafted; §VIII/§X/§XI COHERENCE-REVISED (not silently carried — flagged): §VIII "two normalizers"→THREE (added Loeb DSL column + anchors row); §X citation summary chapter/section→canonical-anchor + fence added (trust follows evidence-tier not storage); §XI kept governance bullets, RETIRED v1 stale status block (points to header + §Governance). Scaffolding removed, section order verified by grep. Status: all sections editor-gated + gap-7 co-ratified; AWAITING whole-draft jurist read + steward ratification of the superseding version. NOT committed (draft; commit on ask). File still -DRAFT / not operative; v1 remains canonical until ratified.
2026-07-03T14:45 — ★ v2.0 RATIFIED — SUPERSEDES v1. Jurist whole-draft read (full 745-line assembly, not sections-alone) + steward ratification. Order-blindness two-layer split praised as "sharper than what I asked for" (the partition I hadn't worked out). Coherence confirmed across the whole doc (storage-vs-evidence in §X right place, §XI status-retirement applies the day's lesson to the doc itself, arithmetic holds). Four items stay correctly open (named not closed = the bar): §II.3 anchor syntax · §IX silence-as-finding · promotion criterion's 2 operational blockers. Realized in artifact: status DRAFT→RATIFIED/OPERATIVE, H1 v2.0.0, supersedes v1. Applied FIRST FIX-class errata (jurist's exactness catch: "on the final-pass review"→exact turn) — the Held-for-Update lane's first real use (jurist had noted nothing used the lightweight path today). REMAINING operational (proposed to steward, not yet done): rename file drop -DRAFT, obsolete-banner on v1, commit (=logchain supersession record). Then corpus pass.
2026-07-03T14:45 — Jurist's two day-level takeaways (worth carrying): (1) MOVE — state a general test, never an enumerated exception list (recurred 3× unprompted: §III portability, PASS-BUT-FALSELY class-gate, gap-7 itself). (2) DISCIPLINE — check the source not the derived view (4 independent catches same root: 74% figure, Loeb page-marker, bad citation in addendum, stale status tag in the doc about stale status). Jurist's honest untested-list: every "settled-in-doctrine-pending-confirmation" empirical gate is named-not-run; three-party model never tested against real disagreement/pushback; FIX-class path unexercised until today's errata. Jurist self-flag: "everything I found was one pass, one perspective — my own derived view."
2026-07-03T15:00 — HOLD FOR SESSION MEMORY (telos-grade, don't lose): the jurist's closing framing of what v2.0's doctrine is, not process — (a) Alexander: the spec is a structure that revised itself through contact with REAL TEXTURE (the actual v1 text, code, DSL) rather than imposed-once-and-static — it has the grain of something grown, which is why it cohered on the whole-draft read; (b) Harrison: Tier-3's refusal to silently correct an author's apparent error (preserve + flag, never fix in canon) is "not letting the living overwrite the dead for the living's own convenience" rendered as a gate. Neither is decoration; both name what the doctrine does. Connects to project-studium-engine-telos-chamber-of-voices + the steward's formation. Belongs in the durable session-memory at wrap.

Confidence to recalibrate

2026-07-03 — RECALIBRATION (honest): my morning map-correction "46 sectionless-valid / 13 loss" was WRONG — the quick break-regex counted frontmatter --- fences AND markdown bullet lists (^\*\s*\*\s*\* spanning newlines) as thematic breaks. Fell into census-through-a-pattern on my OWN correction (the Symmetria §3 flag, same day I relied on it for the classifier). TRUE picture: 9 files have real markdown thematic breaks (auto-rescued by §VII fix); ~50 need per-file judgment split into (a) genuinely-sectionless→declare sectionless:true, (b) • • •/**•**-style separators = a §II.2 break-normalization defect (literal glyph-runs render as prose = no represented structure). The §VII gate fix is CORRECT (asserts "structure represented in markdown"); only my earlier COUNT was contaminated. Lesson re-confirmed: anchor break-detection to the SPEC's break form (---), verify the regex against real files, don't span newlines.

Authorization moves

Sub-agent dialogues

2026-07-03 — Agent C (structure+graduation). audit-agent: PASS (0.95 conf, read every line, greps confirm absence, uncomfortable-not-convenient finding, distinguishes verified/inferred). VERIFIED MYSELF: graduate_to_canonical calls verify_graduation (L63) NOT verify_conversion (0 refs) — only audit_cruft cruft==0 (1 of 5 health checks). git mv no git-add (L133). NO source-archiving (manifest path edits only). graduation-spec.yaml L74-77 DECLARES verify_conversion+verify_graduation+body_word_conservation required → tool violates its own spec. ⇒ the graduation GATE is half-missing: cruft-free-but-damaged files graduate. + verify_graduation itself under-enforces gaps (hardcoded 3 of 6 placeholder patterns). Structure tools healthy (insert_chapter_headings exemplary fail-loud; structure_from_ncx & repair NOT redundant — keep both).
2026-07-03 — Agent A (cleaners). audit-agent: PASS (0.85-0.95, negative-grep rigor, honest empirical-gap caveat). GLOBAL GOOD: NO prophylactic char-normalization in any of 5 (§V Tier-2 honored — the key wave reassurance). strip_cruft (PRIMARY): prose-guard is EXTERNAL/advisory, RESIDUAL>0 prints "don't graduate" then WRITES anyway → port internal word-guard + block-on-residual. clean_pandoc_html_residue: best-guarded, wave-safe. clear_safe_html_residue: orphaned/redundant except → fold+retire. clean_calibre_artifacts: orphaned + real PASS-BUT-FALSELY (strips footnote links→bare numbers→passes guard; predates keep_notes) → retire or add footnote-preflight. clean_pdf_running_heads: hardwired-Auerbach, no guard/backup, blanket ^\d{1,4}$ strip → NOT wave-safe, rename to oneshot + fix runbook L103.

Meta-findings (the class behind the instances — steward asked "what's hiding in plain sight")

2026-07-03 — (1) ONE failure shape fleet-wide: gates enumerate known-bad SIGNATURES not the INVARIANT (normalize_ocr count-not-identity; verify_conversion cruft-signatures→Taylor ^{; verify_graduation 3/6 patterns; audit_cruft signature-list; strip_cruft advisory-residual). Wave-0 fixes must all be assert-the-invariant shape. (2) The fleet has NO self-test — every bug found by reading; only audit_corpus (written today) has --validate. ⇒ build a FIXTURE HARNESS (known-good+known-bad per defect class), do each fix test-first. (3) The gate is a discipline not a door — graduate should be the single fail-closed chokepoint. (4) Checks need ONE home shared by gate+map (apparatus lives only in map now → drift); bigger: build_catalogue could carry live gate-status = standing health ledger (§IX coverage). (5) §V Tier-2 conversion-record written only on OCR path, not EPUB/cleaner path — wider §V gap. Ideas 4-ledger + 5-record → follow-on [HARDENING], not Wave 0.}
2026-07-03 — STEWARD AUTHORIZED all 8 Wave-0 blockers (after this reflection). Method: fixture harness first, then 8 fixes test-first, invariant-not-signature, gate-as-single-door, shared checks. Tool-fleet-compliance ledger written: _curation/tool-fleet-compliance-2026-07-03.md.

Wave-0 progress (test-first, harness = scripts/test_tools.py)

2026-07-03 — BUILT scripts/test_tools.py (the fleet self-test the tools never had; no pytest dep, matches --validate idiom). Each fix pinned by a fixture asserting the INVARIANT it restored.
#5 DONE — verify_conversion §VII sectionless: heading_density passes when 0 headings + real markdown thematic break (---/***) OR sectionless:true; only FLAGS glyph-run separators (• • •) which are a §II.2 break-normalization defect. 3 fixtures green. Recalibrated the map: 9 real-break auto-rescued, ~50 need per-file judgment (not the wrong "46").
#1 DONE — graduate_to_canonical.gate_candidate(): the single door now runs BOTH verify_conversion (health) AND verify_graduation (conventions), imported (no more stdout-parsing subprocess). cruft-free-but-damaged + non-conformant both GATED; healthy+conformant passes. 3 fixtures green. Closes the hole that let 110 damaged files graduate.
#6 DONE — verify_graduation gap check drives from the FULL spec placeholder_patterns list but only inside a gap FRAME (HTML comment / [bracket]); prose false-positives avoided; STEWARD-RULED exempt. 5 fixtures green; green-set regression still passes.
#2 DONE (load-bearing) — normalize_ocr.verbatim_guard now ALWAYS aligns R↔C (merge=concat / doubling=drop-head, per-type counts verified); no more identity-skip when merged_count>0. Catches word-swap-with-balancing-count + unrecorded-join. 4 fixtures green. Signature: (raw,canon,heads,n_merged,n_doubled). Follow-on (noted, not blocker): merge_flag hard-stop policy + wordfreq-venv degradation.
#3 DONE — strip_cruft: in-tool prose-safety guard (never bypassable, delegates verify_conversion.prose_delta) + block-on-residual (--allow-residual override); guards run BEFORE backup (refusal = no side effect). 3 fixtures green.
#4 DONE — clean_epub_residue: FOOTNOTE_RESIDUAL tripwire raises FootnoteResidual before the TOC-unwrap can silently destroy an uncovered footnote family (#filepos/bare-fn); main aborts loud. Real TOC cross-ref doesn't false-trip. 2 fixtures green.
#7 DONE — graduate: git add before git mv (+ plain-move fallback) so untracked batches never silently graduate 0; honest failure count; loud SOURCE-ARCHIVING-NOT-WIRED warning (2026-07-02 directive is a noted follow-on feature-build, not a wave blocker).
#8 DONE — retirements: archived 6 orphaned/dead tools → scripts/archive/ (clean_ocrmac_pagination, clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings, clean_pdf_running_heads[Mimesis one-shot, job done]). FOLDED -block removal into strip_cruft first (capability preserved, fixture-pinned). Fixed runbook's dangerous generic clean_pdf_running_heads reference. No live tool imports an archived one.
ALL 8 WAVE-0 BLOCKERS DONE. Harness scripts/test_tools.py = 21 assertions across 6 tools. Follow-on [HARDENING] (noted, not blockers): source-archiving auto-wire; merge_flag hard-stop + wordfreq-venv pin; §V conversion-record on EPUB/cleaner path; build_catalogue-as-standing-health-ledger (meta-idea 4). NOT committed yet.

Prior-art scans (steward: "incredible we're the only ones"; generative-from-spec principle)

2026-07-03 — Scholarly-encoding scan (audit-agent PASS 0.95, primary sources read). VERDICT: we are NOT first — ~70% of our spec re-derives TEI + CTS + Standard Ebooks. ADOPT: (1) TEI <choice> never-destroy-the-original (orig/reg,sic/corr) — vs our replace-and-sidecar; question for gap-3. (2) TEI ch.13 apparatus <app>/<lem>/<rdg> — anchor to stable LOCATION not footnote NUMBER + verify resolution → hardens gap-2. (3) CTS URN — structural addressing + EDITION-AS-IDENTITY → what §III should become. (4) Standard Ebooks se lint gate (validates our gate arch) + [Editorial]-isolated-commit provenance + per-defect detector scripts (audit_corpus template). (5) Distributed Proofreaders: redundant verification AGAINST THE SCAN (single pass can't ground verbatim); HathiTrust: no ground-truth at scale → quality-tiers (validates map+waves). GENUINELY OURS (keep): checksum byte-integrity (TEI/CTS lack it, ~0.75), LLM-literal-citation constraint, corpus-as-trust-substrate governance. TEI ODD/Roma = spec-generates-validator = the steward's generative principle, already a standard → likely the model. 2 scans still out (tooling/spec→validator, bounded-corpora-for-AI).
2026-07-03 — Tooling/spec→validator scan (audit-agent PASS 0.85, primary sources; the generative-principle answer). TEI ODD/Roma = "One Document Does it All": one spec compiles to BOTH validator AND docs, can't drift = purest generative-from-spec, BUT XML-locked (full adoption = corpus→TEI-XML). VERDICT: realize the principle NOW without full TEI — W3C PROV (PROV-O) = standard vocab for gap-3 conversion-record (adopt, don't invent schema); JSON Schema = free spec→validator for structured layers (.meta.json sidecars, content-types.yml, class enum); PANDOC LUA FILTERS = reimplement cruft/footnote recovery on the parsed AST (structure KNOWN not regex-guessed) → dissolves clean_epub_residue's whole failure family at root; dinglehopper (OCR-D) for GT-diff; ftfy for encoding. GENUINELY OURS (no equivalent anywhere): prose-word-multiset guard (field optimizes plausibility→silent mutation; ours forbids alteration), verbatim-span-in-bounded-source check (RAG does semantic NLI not lexical), checksum integrity. HELD-OPEN FORK [PROPOSAL]: markdown+JSON-Schema now (proportionate) vs TEI-XML+ODD if engine fidelity outgrows markdown — name, don't resolve. 1 scan out (bounded-corpora-for-AI). Then: consolidated synthesis → jurist spec-revision brief (PENDING-46).

32 KiB Raw Permalink Blame History Unescape Escape