session 2026-07-03: doctrine phase complete (Wave 0 + full spec-revision rulings + Loeb substrate reframe); PENDING-46; Phase 2 v2.0-draft next
This commit is contained in:
David F Glidden
+102
@@ -1023,3 +1023,105 @@ The Jurist's six-phase authorization map governs sequencing. Cross-cutting succe
|
||||
**Out of scope:** Crawford's `filepos` positional family (ref/def don't share a stem) — a different pairing model, still owed; documented in the tool docstring.
|
||||
**Files affected:** `scripts/clean_epub_residue.py` (Family D); `canonical_texts/…/environmental/the-craftsman-sennett-richard.md` (Sennett reconverted + spec-normalized headings); `catalogue.yaml`; `_curation/tool-evolution-log.md`.
|
||||
**Awaiting:** steward acknowledgement (a tool addition within the authorized "build the tool the batch needs" discipline; recorded for the paper loop alongside PENDING-44).
|
||||
|
||||
## PENDING-46 — Spec gaps surfaced by the corpus-fidelity work + the generative-from-spec principle
|
||||
**Date:** 2026-07-03
|
||||
**Tag:** [PROPOSAL] (gaps 1–6) + [ESCALATE] (gap 7, generative principle as governance)
|
||||
**Summary:** Two days of corpus-map + tool-fleet work surfaced 7 gaps between the chamber-library spec (§§I–XI) and lived experience; steward named the governing principle (Spec Loom / hic jacet): tools should be GENERATIVE from spec, marked once, mutable-after — not hand-carved beside it.
|
||||
**The gaps (drafts for the jurist/steward loop; see _curation/tool-fleet-compliance-2026-07-03.md + this session):**
|
||||
1. §II.2 break-form unenforced; `sectionless: true` field is READ by the fixed verify_conversion but NOT defined in the spec (tool got ahead of the constitution). Define the field; gate the `---` invariant; name `• • •`→`---` normalization.
|
||||
2. §V apparatus-integrity asserted but never gated (70 damaged files). Make ref↔def pairing / no-stranded-notes a gated invariant; the audit_corpus classifier is the instrument; belongs in the graduation gate.
|
||||
3. §V Tier-2 conversion-record required but only the OCR path writes one; mandate it for every normalizing path + define the schema (the grounding gate needs the declared equivalences).
|
||||
4. §VII [PARTIAL] → formalize: the declared gate-list (graduation-spec.yaml gates[]) IS the gate; fold in sectionless exception + apparatus check + body_word_conservation (declared, unimplemented).
|
||||
5. §IX [OPEN] → closeable now: audit_corpus map + build_catalogue-as-health-ledger = the coverage-warrant.
|
||||
6. The STANDARD (verified fidelity = floor not tier) + the FENCE (green set) live in memory, not the spec; distinguish converted/verified/fenced.
|
||||
7. [ESCALATE] the generative principle as governance: tools are thin consumers of the machine-readable spec; declared convention-data + gate-list authoritative; a tool re-encoding a convention is drift. Reshapes how we build — jurist+steward to ratify.
|
||||
**Prior-art dependency:** 3 research scans running (TEI/scholarly encoding · fidelity tooling & spec→validator · bounded-corpora-for-AI) — TEI likely hands us a ratified apparatus/provenance/normalization vocabulary to adopt INTO the spec rather than invent. Synthesis owed before drafting the spec revision.
|
||||
**Recommendation:** mark-the-ground sequence — consolidate spec (close 1–6, rule 7) → build generative validator (spec→spec-lint, single gate) → THEN run cleaning waves. Waves-first would bake tool-shaped conventions into 300 files (the redo we're avoiding).
|
||||
**Files affected (on authorization):** docs/chamber-library-specification.md, _curation/graduation-spec.yaml (extend as the convention-data source), a new spec-lint validator.
|
||||
**Awaiting:** jurist doctrine pass + steward authorization; prior-art synthesis.
|
||||
|
||||
### PENDING-46 addendum (2026-07-03) — prior-art synthesis complete
|
||||
Three scans done → `chamber-library/docs/spec-revision-prior-art-synthesis-2026-07-03.md` (the jurist brief).
|
||||
KEY: ~70% of our spec re-derives TEI + CTS + Standard Ebooks (align, don't invent). The FUSION
|
||||
(verified-verbatim fidelity of a bounded curated corpus as a constitutional invariant of an
|
||||
epistemic-memory substrate) is close-to-novel (provisional). WATCH: Seck "Governed Reasoning for
|
||||
Institutional AI" arXiv 2604.10658 (independent convergence on our governance half; lacks our
|
||||
corpus-substrate moat) → landscape register. ALIGNMENTS: §III→CTS URN (edition-as-identity);
|
||||
§V apparatus→TEI ch.13 (anchor-to-location, verify resolution); conversion-record→W3C PROV;
|
||||
§VII→ODD one-artifact discipline; §IX→ledger-as-coverage-warrant. GENERATIVE PRINCIPLE realizable
|
||||
NOW proportionately (PROV + JSON-Schema + Pandoc Lua filters + ODD-as-model); TEI-XML/ODD fork
|
||||
HELD OPEN (name, don't resolve). ADOPT: dinglehopper, ftfy, JSON-Schema, Lua filters, SE
|
||||
[Editorial]-commit, Deterministic-Quoting (engine citation path). KEEP (no equivalent anywhere):
|
||||
prose-word-multiset guard, verbatim-span check, checksum integrity. TWO held-open architectural
|
||||
questions for the jurist: (a) in-band original (TEI <choice>) vs replace-and-sidecar; (b) the
|
||||
markdown-vs-TEI-XML fork. SEQUENCE: jurist spec-revision → build generative core → THEN waves.
|
||||
|
||||
### PENDING-46 — JURIST RULING RECEIVED 2026-07-03 (record: chamber-library/docs/spec-revision-RULING-2026-07-03.md)
|
||||
Steward+jurist still discussing gap 7. Executor to act ONLY within ratifications below when steward authorizes execution (spec is constitutional; ruling ≠ steward's go-ahead to edit the spec doc).
|
||||
|
||||
**THE UNIFYING INSIGHT (jurist's condition, load-bearing):** gaps 2 (apparatus ref↔def anchors), 3 (PROV "original" as resolvable reference), and item-8 (§III CTS-URN citation) ALL reduce to ONE logical-passage addressing/pointer substrate — "don't build two addressing systems that can drift." DESIGN THE POINTER SUBSTRATE ONCE; three consumers. This is do-it-once made concrete.
|
||||
|
||||
**READY TO EXECUTE on steward's word (RATIFIED-CONDITION):**
|
||||
- Gap 1: define `sectionless: true` + NAME AN ATTESTOR FIELD (who set it / against what evidence the SOURCE lacks breaks) — positive attestation, never a default/absence state.
|
||||
- Gap 2: apparatus-integrity gated invariant; anchors on the shared addressing substrate.
|
||||
- Gap 3: PROV conversion-record, MANDATORY on every normalizing path; "original" = resolvable reference (survives question-a either way); + name the gate's TWO functions distinctly — content-grounding (canonical-text-only, §VI invariant intact) vs fidelity/process-integrity (reads the record). Extending the gate to read the record does NOT violate sidecar-exclusion.
|
||||
- Gap 4: spec-text "declared gate-list IS the gate" now; body_word_conservation impl tracked SEPARATELY (open eng).
|
||||
- Gap 5: §IX ledger-as-coverage-warrant + FRESHNESS INVARIANT built in from the start (regen inside the gate pipeline OR checksum tying entries to file state).
|
||||
- Item 8: §III its OWN section (CTS-URN, edition-as-identity); shares substrate with gap 2, distinct question (cites OUT vs internal ref↔def).
|
||||
- Question (a) PARTIAL-DECIDE: extend the grounding gate to READ+CHECK the conversion record NOW (closes the fidelity blind spot); in-band-vs-separate-file left open (now a low-stakes usage question).
|
||||
|
||||
**NEW PENDING ITEMS the ruling created (NOT executor-actionable yet):**
|
||||
- (i) Gap-6 promotion criterion — the mechanical converted→verified test. ESCALATED. No file may be promoted to "verified" until named+checkable. (Jurist: cite Seck's "governability" axis — knowing when NOT to treat a file as verified — directly when writing this.) Tiers (converted/verified/fenced) themselves RATIFIED.
|
||||
- (ii) body_word_conservation implementation (gap 4's unfinished half).
|
||||
- (iii) Gap-7 amendment-process corollary: ratifying "spec is the generative source" concentrates authority → WHO can amend chamber-library-specification.md and under what process must be decided IN THE SAME BREATH as gap 7. Returned to steward+jurist.
|
||||
|
||||
**Question (b):** HELD; trigger = revisit if/when (a) resolves toward full in-band dual-content at scale.
|
||||
**Seck arXiv 2604.10658:** jurist-confirmed real; add to landscape register. Adjacent cluster (Pierucci "Institutional AI" 2601.10599/2601.11369; "Governing Actions Not Agents" 2606.26298 — an "adjudication branch" ≈ steward/jurist/executor) → jurist wants a REAL SWEEP before gap-7 ratification. Executor launching that sweep now to feed the live gap-7 discussion.
|
||||
|
||||
### PENDING-46 — gap-7 amendment-process sweep DONE 2026-07-03 (→ chamber-library/docs/gap7-amendment-process-JURIST-ADDENDUM-2026-07-03.md; copy on Desktop for handover)
|
||||
FINDING 1: our model's ORGANS are precedented (2601.11369 Manifest/Oracle/Controller ≈ jurist/adjudication/executor+logchain; 2606.26298 executor-gate; Seck 2604.10658 tiers+ledger) BUT the closest analogue LEAVES AMENDMENT UNDEFINED — the exact trap the ruling steers around. Distinctive to us (0.7, no-counterexample): human-steward-load-bearing + non-implementing-jurist + fidelity-verified-corpus-substrate. Moat holds (0.75): none couple governed-reasoning to a verbatim-fidelity corpus.
|
||||
FINDING 2: the amendment process nearly writes itself from prior art we ALREADY run — (1) RFC immutable-supersession = logchain-on-the-spec; (2) BIP/PEP "normative-change-reverts-to-Review" = the crisp FIX→PROPOSAL trigger, roles map 1:1; (3) semver-as-consumer-contract = spec-change-is-API-change, MAJOR re-runs the verification ladder on every consumer. TEI Council = domain twin (decision-in-ticket ≈ PENDING/REVIEWED). Anthropic constitution = cautionary (no codified amendment procedure). Failure mode: single authoritative artifact + no amendment process → capture/ossify/fork; generative principle raises stakes on all three.
|
||||
A synthesized draft amendment process is in the addendum for the jurist to rule on. Executor holds — gap 7 + amendment process are steward+jurist's to co-ratify.
|
||||
TODO (executor, standing offer): add Seck 2604.10658 + this cluster (2601.10599/11369, 2606.26298) to the landscape register; commit Wave-0 changeset (still uncommitted).
|
||||
|
||||
### PENDING-46 — gap-6 EVIDENCE-TIER DATA produced 2026-07-03 (jurist asked; → docs/gap6-evidence-tier-data-JURIST-2026-07-03.md; Desktop copy)
|
||||
Joined corpus-quality-ledger ⋈ source-matches.json. EVIDENCE TIERS (verify-against-what): V-DSL(Loeb)=952 [930 pass floor], V-TEXT(born-digital source)=239 [6 pass], V-SCAN(pdf)=50 [0], V-SUSPECT=29, V-NONE=14. HEADLINE: 1191/1284 (93%) DETERMINISTICALLY verifiable (V-DSL+V-TEXT) — source IS ground truth, our prose-word-guard IS the promotion test; only 50 (V-SCAN) need the hard OCR-vs-scan path. 936 already pass floor; 255 more verifiable after the waves clear their floors. Proposed criterion tiers: verified-deterministic (word-identical re-convert vs source) / verified-scan (OCR-vs-image) / fenced-out (V-SUSPECT confirm-match + V-NONE acquire-source = Seck governability). CAVEATS: (1) add evidence-tier as a STANDING ledger column (couple gap-6 to gap-5); (2) source-match ~1% author-agreement FP → V-SUSPECT is a real gate; (3) deterministic tier COUPLED to item-8 edition-as-identity (wrong-edition match fails word-guard). The "no ground-truth at scale" worry that escalated gap-6 shrinks to 50 files.
|
||||
|
||||
### PENDING-46 — GAP-6/7 CONSOLIDATED RULING LANDED 2026-07-03 (record: chamber-library/docs/spec-revision-RULING-2026-07-03.md is the ORIGINAL; consolidated ruling supersedes gap6/gap7 per RFC-style; steward relayed)
|
||||
RATIFIED: gap-6 promotion criterion (3 tiers; tier-1 amended to require source-identity author+edition via item-8, then word-identical re-convert); gap-7 generative meta-principle (no tool may encode a convention independently of the declared spec); amendment process (RFC supersession + change-class + semver + PENDING/REVIEWED + no-new-organ). CONDITIONS: (6) keep V-SUSPECT & V-NONE DISTINCT ledger values; (7a) editorial/normative test must directly ask "does this change what any current gate accepts/rejects" (not the FIX label / declared-field-touch); (7b) deprecate-before-remove extends to PROMOTED FILES — migration-window policy for files verified/fenced under a prior criterion (the ~1% author-agreement FP ~dozen files is the real population). This consolidated ruling is itself the FIRST instance of the amendment process (supersession).
|
||||
JURIST DIRECT Q — ANSWERED: Loeb IS edition-unambiguous by construction (single loeb.dsl, 1 entry/work, Loeb Classical Library = the fixed edition; no which-edition question) → the 930 floor-passing V-DSL do NOT need item-8 for verification; only the 239 V-TEXT (matched via source-matches = real edition risk) do. Item-8 gates V-TEXT, not V-DSL.
|
||||
⚠ NEW FINDING (surfaced, qualifies "930 ready now"): ALL 952 Loeb retain §II-forbidden '— p. N —' page-markers as h6 headings; 930 "pass" the floor but verify_conversion BLIND-SPOTS them (markers are markdown headings, not cruft signatures) → they are NOT §II-clean. This is (a) a §VII/gap-4 GATE GAP (the gate doesn't enforce §II 'no page markers'), and (b) a DOCTRINE question for the jurist: strip the Loeb page-markers (§III off-canon) OR grant a Loeb page-reference exemption (classical texts cite by canonical reference). So "930 ready now" → "930 pending a Loeb page-marker disposition." Contamination-directive: the gate's "930 clean" self-assessment is not fully honest re §II; named now.
|
||||
CITATION-INTEGRITY LESSON (jurist §3 caught): addendum cited arXiv 2605.24229 for Anthropic "no codified amendment procedure" — that paper is an empirical compliance audit, doesn't support the claim. Let a sub-agent's blanket "4/4 verified" ride without per-claim check. DISCIPLINE: verify each citation against the specific claim it carries; never let aggregate "verified" substitute. → feedback memory candidate.
|
||||
|
||||
### PENDING-46 — THREE-QUESTIONS RULING + executor factual check 2026-07-03
|
||||
QUESTION A RESOLVED → STRIP branch. Factual check (verified): '— p. N —' = Loeb PRINT-RUN page numbers (plain sequential integers) = edition-specific → strip per §II. Stephanus/Bekker margin NUMBERS are ABSENT from the DSL extraction (read Plato Charmides body — the canonical Stephanus case — none present; the "309 Stephanus-shaped" signal = apparatus-citation noise "Eclogae 4.24b"/"Athenaeus 6.224c", NOT margin refs). Stephanus subsection LETTERS may survive mangled inline (stray "B" fused to Greek word) — separate cleanup obs. CONSEQUENCES: (1) Loeb tier (74% of corpus) currently lacks a standard citation anchor → cite by intrinsic structure OR work+verbatim-quote (§III sectionless reduction, already sanctioned) = ACCEPTABLE; sourcing Stephanus/Bekker = separate FUTURE enrichment, not this strip, noted in §III/item-8. (2) Loeb-tier STRIP PASS (952 files) needed before the 930 are §II-clean/promotable — mechanical wave, still deterministic (re-extract DSL + strip + word-guard). (3) Gate rule keys on a PER-GENRE recognized-locator REGISTRY (Stephanus/Plato, Bekker/Aristotle…), NOT the word "Loeb"; registry empty for Loeb now → all print-pages flagged→stripped.
|
||||
QUESTION B RATIFIED: tier-2 = REDUNDANT independent verification (≥2 human passes vs scan image + 3rd adjudicating; dinglehopper = inter-pass diff; OCR-confidence = triage only, NOT the test). Numeric bar DEFERRED to a 3-5 file calibration batch. (Executor owned the CER/WER-vs-image conceptual error — no ground truth for V-SCAN by definition.)
|
||||
QUESTION C RATIFIED: re-verify-pending = own ledger status; verified_under:<criterion-semver> per row; N=2 release cycles PROVISIONAL (set vs real cadence, keep short — protects future willingness to tighten); + logged FIX-class extension path (operational delay ≠ epistemic failure).
|
||||
ALL THREE now fold into the v2.0 draft: A→§II strip rule via per-genre locator registry + §III/item-8 Loeb-anchor-absent note + Loeb strip wave; B→gap-6 §2 tier-2 mechanism (bar [OPEN-calibration]); C→amendment §7b migration policy.
|
||||
|
||||
### PENDING-46 — LOEB SUBSTRATE REFRAME 2026-07-03 (steward sent me to the DSL source — decisive)
|
||||
FINDING (verified against loeb.dsl source): the canonical citation anchors are IN THE SOURCE, FLATTENED by extraction — NOT absent. Plato Stephanus = `[c dimgray]St. II p. 153[/c]` + subsection letters B/C/D/E (the "stray B" earlier); Homer = book-letters `[b]A[/b]` + line-numbers `[c dimgray]5/10/15`; Loeb pages = `[c dimgray]Page number N` (246,622 marks). The `[c dimgray]` class is OVERLOADED (≥5 semantic uses); extract_loeb_dsl.py's KNOWN_TAG.sub("") strips the wrapper → collapses all into undifferentiated inline text. Stephanus SURVIVED into the extract as bare "St. II p. 153" (my grep searched the wrong form "153a" — 3RD census-through-a-pattern miss this session).
|
||||
REFRAME: Loeb path is NOT "strip page-markers" and NOT "external enrichment/acquisition" — it's RE-EXTRACT preserving the [c dimgray] semantic layer, disambiguated BY CONTENT SHAPE (per-genre locator registry = the disambiguation map): Page-number→off-canon(§III); St./Bekker/line→citation anchor(item-8/CTS); work-title→drop; [sup]→apparatus. My "74% cite-by-work+quote" was WRONG; Loeb can be properly cited by canonical anchors → feeds item-8 CTS directly (Plato-by-Stephanus is textbook CTS). The jurist's "structured-but-unanchored→work-level fallback" shrinks to a small residual; "external acquisition" enrichment case near-EMPTY.
|
||||
META-LESSON (3× this session, steward-corrected 2×): for a fidelity corpus, verify structure/anchor claims against the AUTHORITATIVE SOURCE, never the derived artifact or a regex over it. → feedback memory + Symmetria §3 flag candidate: derived-artifact-or-regex-over-substrate.
|
||||
DECISION (steward delegated "whatever you think best"): this changed a FACTUAL PREMISE the jurist's §III ruling rested on → must go back to jurist to re-rule, but on VERIFIED DATA not my prose (3× wrong). So: launched a careful per-genre DSL anchor audit (fresh agent, anti-pattern-drift discipline, verify-against-real-cards, quantify the genuine residual) → I verify its load-bearing claims against the source → relay reframe+audit to jurist for §III re-ruling → THEN build the re-extraction on the ratified disambiguation map. Audit running (agent ae7ad9b).
|
||||
|
||||
### PENDING-46 — LOEB FULL-RUN SCOPE (steward recall 2026-07-03)
|
||||
The current Loeb corpus (~952 work-files) is TIER 1 ONLY of the 3-tier plan `chamber-library/_curation/loeb-canon-proposal-2026-06-15.md` (steward's frame: "begin highest-signal now, comprehensive next, all of it once dilution is solved"). Tier 1 = 86 authors/356 vols/81,912 cards (67%); Tier 2 = 151 authors/92%; TIER 3 = ALL 377 authors/523 vols/121,447 cards ("waits on dilution").
|
||||
STEWARD CALL (do-it-once): since we're rebuilding extract_loeb_dsl.py anyway to preserve the citation structure, run the FULL DSL (all 377 authors) in ONE pass — not Tier-1-then-redo.
|
||||
DILUTION RESOLVED BY THE FENCE: Tier 3's "waits on dilution" condition is now MET by gap-6's fence — extract everything citation-rich, fence the ENGINE to the verified/green set. Corpus-completeness ≠ engine-scope (same don't-conflate-axes discipline). Full extraction and dilution no longer in tension.
|
||||
⇒ Re-extraction scope = FULL DSL (377 authors), not just the 952. The running per-genre anchor audit (agent ae7ad9b) is already whole-DSL-scoped → its disambiguation map serves the full run. One rebuilt extractor, run once, recovers clean bilingual text + Stephanus/Bekker/book+line anchors → feeds §II, §III/item-8, gap-6 together.
|
||||
|
||||
### PENDING-46 — LOEB ANCHOR AUDIT COMPLETE + VERIFIED 2026-07-03 (→ chamber-library/docs/loeb-anchor-audit-and-disambiguation-map-2026-07-03.md)
|
||||
Per-genre DSL audit done (agent, 22min, whole-DSL); executor RE-VERIFIED the two load-bearing claims against source. Scale: full DSL = ~2,041 works / ~124,627 cards (current 952 = Tier 1). Anchors richly present per-genre (Plato Stephanus / Aristotle Bekker-sparse+chapter+section / Homer book+line / tragedy line+strophe / orators section+col / historians book.ch.sect / Presocratics Laks-Most D/R/P PLAIN-TEXT / fragments number) — all verified against real cards + my own Plato/Homer/Aristotle/Presocratic spot-checks. RESIDUAL (the number §III re-ruling turns on): 37 machine-upper-bound → true substantive-anchorless floor SINGLE DIGITS, and those are fragment/testimonium/pattern-poem/concordance artifacts where work-level IS correct granularity. ⇒ >99% of works carry a recoverable anchor. RECOVERY-CRITICAL VERIFIED: the dropped `%` index lines carry Book/Letter/Fragment divisions — 35,261 such lines, thrown away by the current extractor. Disambiguation routing map captured (the design input for the rebuilt extractor).
|
||||
WHAT IT MEANS FOR §III RE-RULING (SIMPLER than feared): the jurist's "structured-but-unanchored→work-level fallback" normative extension is NEARLY MOOT — population is single-digit genuine fragments (original sectionless reduction covers them, not a new triggering condition). "External enrichment/acquisition" case is EMPTY. §III cites the Loeb by RECOVERED canonical anchors (CTS edition-as-identity: Plato-by-Stephanus etc.). Re-extraction recovers clean text + anchor layer + work-divisions in one pass over the full 2,041-work DSL.
|
||||
NEXT: relay verified audit + this simplification to the jurist for the §III re-ruling; then rebuild extract_loeb_dsl.py to the disambiguation map + run full DSL.
|
||||
|
||||
### PENDING-46 — §III CITATION POSTURE RULING (jurist, 2026-07-03) — DOCTRINE PHASE COMPLETE
|
||||
DISPOSITION:
|
||||
- structured-but-unanchored extension: VOID as ratified-for-74% (premise gone, not smaller — a principle that now governs NOTHING shouldn't stand as active doctrine = "gone decorative"). Retained ONLY as an explicitly DORMANT, named contingency (future-acquired work with real internal structure + no recognized anchor convention), LABELED in the spec as inapplicable to any work currently in the corpus. Not deleted (keep the reasoning), not left active.
|
||||
- "future enrichment" trigger requirement: RETIRED — no external-acquisition case remains (everything, incl. the 35,261 index-line divisions, is IN the DSL → re-extraction not acquisition).
|
||||
- §III citation posture: RATIFIED — cite the Loeb by its RECOVERED canonical anchor, resolved by edition-as-identity; work-level fallback ONLY for the genuine-fragment residual. Doctrine now; recovery = downstream engineering, validator-gated, feeding gap-6.
|
||||
- Per-genre mapping/disambiguation table: NOT RATIFIED — re-extraction ENGINEERING to review once built (jurist hasn't seen the audit, only the summary; the ambiguous string-match calls resolve during extraction). The architecture above the table is sound regardless. → §III's per-genre registry is a LIVING data layer (grows/reviewed during extraction), NOT ratified doctrine — matches the doctrine-stable/data-living split I described to the steward.
|
||||
- STANDING PROCESS NOTE (jurist, not corpus-specific): ANY corpus-population claim feeding spec doctrine must be SOURCED FROM THE DSL/SOURCE DIRECTLY, or explicitly flagged extract-derived+provisional, BEFORE it does normative work. Same discipline gap-2/gap-3 enforce on the corpus, applied to how the corpus is AUDITED. → adopt as standing practice: verification-ladder entry + Symmetria §3 flag + feedback memory (converges with my own 3×-this-session verify-against-substrate lesson). Capture at wrap.
|
||||
⇒ §III was the LAST gating ruling. DOCTRINE PHASE COMPLETE — all gaps 1-8, generative principle, amendment process, three questions, §III posture ratified. Phase ② (draft spec v2.0 superseding version + generative validator) is UNBLOCKED. Remaining OPEN-marked (non-blocking): per-genre table (engineering), tier-2 numeric bar (calibration), the two held forks (in-band/sidecar; markdown/TEI-XML).
|
||||
|
||||
@@ -51,7 +51,10 @@ permalink: claude-memory/memory
|
||||
- [Be (laundromat)](project-be-laundromat.md) — canonical workstream tracker established 2026-06-08 (Seb-relay of locked decisions). Be = Skemantix startup (Seb+David) funding CapableMind's funding-ladder; **bridge, not venture**. Decisions LOCKED: entity/exit (CapableMind decoupled, grant-funded), pricing (Living $12.99/mo · Archive $69.99/yr · Memorial $49.99/yr · Renovate ~$199 · $8.99 floor), CF Self-Serve Agency + versioned-template-package infra. **a11y gate MERGED (Pat 100/100/100).** Pre-revenue: the WTP gate = renovate Pat → charge her. **Discipline: stop adding spec until the gate clears → nothing for executor on be until then.** Repo @ `f43a0fd`.
|
||||
|
||||
## Active Session
|
||||
- [Session 2026-07-02 evening — Sennett + Family-D tool + the corpus standard & method](session-2026-07-02-evening-corpus-standard-and-the-method.md) — Short evening session; ONE work graduated but the value was elsewhere. **Reconverted+graduated Sennett *The Craftsman*** (calibre-era canonical had silently-lost note-linkage + old-schema frontmatter + `****` residue) — recovered 294 endnotes, spec-normalized headings (incl. a setext Conclusion), both gates PASS, verbatim-preserved; **committed `10cff59`**. To do it, **built+tested+committed the Family-D (stem-suffix) footnote handler** in `clean_epub_residue.py` (regression caught+fixed against Families B/C) — the 4th of ≥5 anchor families the corpus actually uses (Crawford's `filepos` = a different pairing model, still owed). **PENDING-45.** Then a **corpus-quality census** answered the steward's "is poor-quality-graduated widespread?": **YES, 3 distinct axes** — conventions **8/332 pass** (pre-spec, mechanical), text-health **110/332 damaged** (OCR'd big works: Bringhurst/Hochuli/Taruskin/music-bios), **apparatus UNMONITORED** by either gate (Sennett proves "has notes" ≠ "apparatus intact"). Steward felt the vision slipping (**"mandelbrot set… whack-a-mole… clutching sand… I tremble at making the engine work"**) → named the **STANDARD** (verified verbatim fidelity + provenance + apparatus for ANYTHING the engine cites — a floor not a tier; "graduated"=converted+filed ≠ verified; a quietly-corrupt bounded corpus is WORSE than none; fence the engine to green NOW, grow green→100% by waves) and the **METHOD** (whack-a-mole = working reactively per-file; cure = **measure-the-whole-first → one quality MAP → remediate BY DEFECT-CLASS in finite waves, each a countdown**). **PULLING THREAD: build `audit_corpus.py` + the apparatus check → the full quality ledger/MAP of all 1,284 works (the instrument that ends the whack-a-mole), then plan waves from data. Do NOT resume graduating reactively.** Q: did the map get built + real defect-class distribution + is Wave A (frontmatter migration, ≈8→200) the right first countdown?
|
||||
- [Session 2026-07-03 — doctrine phase complete; Phase 2 next](session-2026-07-03-doctrine-phase-complete-phase2-next.md) — **A very long session that closed the entire DOCTRINE PHASE of the chamber-library spec revision.** Morning: built `audit_corpus.py` + the corpus quality MAP (delivered yesterday's thread). Then steward challenged the framing → **tool-fleet audit (3 agents) → WAVE 0**: fixed all 8 blocking tool bugs test-first (`scripts/test_tools.py`, 21 assertions; graduation gate was HALF-MISSING; normalize_ocr guard off in common case; etc.), retired 6 dead tools, **committed `f9cbb8e`+`1c88650`**; record `_curation/tool-fleet-compliance-2026-07-03.md`. Then **prior-art scans (3 agents)**: ~70% of our spec re-derives **TEI + CTS + Standard Ebooks + W3C PROV + TEI-ODD** — adopt their marked ground; watch **Seck arXiv 2604.10658** (independent governance convergence, added to landscape-register). Then **the FULL spec revision RULED by the jurist** (via steward relay): gaps 1-8, generative principle, amendment process (RFC-supersession + change-class + semver), the three questions (A/B/C), and §III citation posture — **ALL RATIFIED**. Unifying insight: gaps 2+3+item-8 = ONE addressing substrate. Then **THE LOEB SUBSTRATE REFRAME** (steward: "look at the DSL directly" — decisive): I'd told the jurist 74% of the corpus was anchorless; the SOURCE inverted it — Stephanus/Bekker/book+line anchors are ALL in `loeb.dsl`, **FLATTENED by extraction** (`extract_loeb_dsl.py:47`). Per-genre audit (re-verified vs source): **>99% of ~2,041 works carry a recoverable anchor**; 35,261 division-carrying `%` lines discarded; residual = single-digit fragments. §III re-ruled → cite the Loeb by RECOVERED canonical anchor (CTS edition-as-identity); the structured-but-unanchored extension VOIDED (dormant only). **Full-run scope decided:** current 952 = Tier 1; rebuild-anyway → run the FULL DSL (2,041 works); fence discharges dilution. Installed wordfreq (venvs) + jsonschema. Disambiguation map durable: `docs/loeb-anchor-audit-and-disambiguation-map-2026-07-03.md`. **THE ARC:** yesterday's despair ("clutching sand") → a marked, finite plan (ground → instruments → corpus → fence). **MY drift-of-the-session (jurist-elevated to standing practice): trusted a derived-artifact/regex over the SOURCE 3× — the steward twice sent me to the substrate and was right.** **PULLING THREAD: Phase 2 — draft `chamber-library-specification.md` v2.0 (superseding, semver-MAJOR) + the generative validator, BEGINNING with a clean END-TO-END READ of the current spec (deliberately NOT read today — I worked from audits), then §III the CTS addressing spine.** Everything in PENDING-46. Q: does the real spec text hold anything the day's audits missed/collide-with, before v2.0 is drafted against my model rather than the text?
|
||||
|
||||
## Archived (2026-07-03 — doctrine phase complete)
|
||||
- [Session 2026-07-02 evening — Sennett + Family-D + corpus standard & method](session-2026-07-02-evening-corpus-standard-and-the-method.md) — Graduated Sennett *The Craftsman* (294 endnotes recovered); built the Family-D footnote handler (`clean_epub_residue.py`); committed `10cff59`; PENDING-45. Corpus census → steward named the STANDARD (verified-not-just-graduated; fence engine to green) + METHOD (measure-whole-first → map → remediate-by-defect-class in finite waves). Led directly into 2026-07-03's map + the full spec-revision doctrine phase.
|
||||
|
||||
## Archived (2026-07-02 evening — corpus standard & the method)
|
||||
- [Session 2026-07-02 — Chamber source-provenance archive + pipeline](session-2026-07-02-chamber-source-provenance-archive.md) — Woke toward the Making graduation queue; a second Jonas copy opened a full source-provenance arc. **Jonas p248 dissolved** (the "illegible page" held only the "Notes" divider — no content lost). **Built the Chamber Sources archive** (`~/Library/Mobile Documents/com~apple~CloudDocs/Chamber Sources/`, iCloud Keep-Downloaded): **322 verified source COPIES, 3.3 GB** + 14 Braid/L2 in `pending-graduation/`. Covers ~all of the ~332 non-Loeb corpus (1284 canonical; 952 = Loeb, pipeline-sourced/excluded). **Built + committed the pipeline (`7553f59`, pushed):** `build_inventory.py` (reproducible census — Apple Books + unzipped .epub dirs + .zip; the gaps that hid ~25 works), `match_sources.py` (internal-metadata + diacritic-fold + TITLE-coverage scoring + Loeb-skip + author-agreement VERIFICATION; 321/335 non-Loeb confirmed), `archive_sources.py`; runbook `source_provenance:` section. **Crux:** first census excluded Apple Books + accent-blind greps → wrongly called ~25 works "absent" + posed FALSE edition-dilemmas; steward corrected *"the texts exist because I have them"* — every dilemma dissolved to the correct edition once Apple Books included. **Caught+fixed 3 high-conf FPs** (meditations→Daily-Laws→Hays; aesthetics/Adorno→Sontag→Adorno; nietzsche/Zweig→Nietzsche's-Works→Zweig). Steward directive: **source-archiving is now a STEP OF graduation.** **PULLING THREAD: come through the graduation queue (~12 Making + 14 Braid/L2), filing each source as we graduate.** Q: did the short evening session happen + which source graduated first?
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: Session 2026-07-03 — the doctrine phase (Wave 0 → full spec-revision rulings → Loeb substrate reframe); Phase 2 next
|
||||
description: "A very long session that went corpus-quality-map → tool-fleet audit (Wave 0, 8 fixes committed) → prior-art scans → the FULL chamber-library spec-revision ruled by the jurist (gaps 1-8, generative principle, amendment process, three questions, §III citation posture — ALL RATIFIED). Then the steward sent me to the loeb.dsl SOURCE and it inverted a premise: the Loeb canonical anchors (Stephanus/Bekker/book+line) are PRESENT in the source, FLATTENED by extraction — not absent. Full-run scope decided (all 2,041 works, fence resolves dilution). DOCTRINE PHASE COMPLETE. PULLING THREAD: Phase 2 — draft chamber-library-specification.md v2.0 (superseding, semver-MAJOR) + the generative validator, beginning with a clean END-TO-END READ of the current spec (deliberately NOT read today — I worked from audits), then §III the CTS addressing spine. Wrapped to clear context for a fresh, focused Phase-2 start."
|
||||
metadata:
|
||||
type: project
|
||||
originSessionId: 019pnD8uNFC9Q2dbLQa3MBDc
|
||||
---
|
||||
|
||||
# Session 2026-07-03 — the doctrine phase; Phase 2 (build the ground) is next
|
||||
|
||||
Started at yesterday's pulling thread — build `audit_corpus.py`, draw the corpus quality MAP, end the
|
||||
whack-a-mole. Did that in the first hour, then the day opened into something much larger and closed
|
||||
the entire **doctrine phase** of the chamber-library spec revision. The everything is in `~/PENDING.md`
|
||||
PENDING-46 (the blow-by-blow governance record) — this file is the working state + the thread.
|
||||
|
||||
## PAST — what we did + why
|
||||
|
||||
**The map (the morning's thread, delivered).** Built `chamber-library/scripts/audit_corpus.py` +
|
||||
`_curation/corpus-quality-ledger.tsv` (1,284 rows) + the interpreted `corpus-quality-map-2026-07-03.md`.
|
||||
Distribution: Loeb 930/952 clean, non-Loeb 8 clean / 324 A-convention / 110 B-textdamage / 70 C-apparatus
|
||||
(the NEW apparatus axis). Answered yesterday's literal question.
|
||||
|
||||
**The steward challenged the framing → the tool-fleet audit → WAVE 0.** "Are the TOOLS spec-compliant
|
||||
before we run a wave? The tool IS the wave." 3 parallel agents audited the fleet. Found: the graduation
|
||||
gate was HALF-MISSING (never ran verify_conversion), normalize_ocr's guard was off in the common case,
|
||||
strip_cruft wrote even on residual, clean_epub_residue silently destroyed unrecognized footnote families,
|
||||
13/15 tools standalone. **Fixed all 8 blockers, test-first** (`scripts/test_tools.py`, 21 assertions),
|
||||
retired 6 dead tools → `scripts/archive/`, folded `<nav>` into strip_cruft. **Committed f9cbb8e** + docs
|
||||
1c88650. Full record: `_curation/tool-fleet-compliance-2026-07-03.md`.
|
||||
|
||||
**Prior-art scans (3 agents) → "we are not the only ones."** ~70% of our spec re-derives TEI + CTS +
|
||||
Standard Ebooks (15-35 yrs old). Adopt: TEI `<choice>` (never destroy original), TEI ch.13 apparatus
|
||||
(anchor to LOCATION not footnote-number), **CTS-URN** (edition-as-identity — §III's shape), W3C PROV
|
||||
(the conversion-record), TEI ODD (spec→validator = the generative principle, XML-locked → held fork),
|
||||
Standard Ebooks `se lint` + `[Editorial]`-commit. Genuinely ours: the prose-word-multiset guard,
|
||||
verbatim-span-in-bounded-source check, checksum integrity. Watch: **Seck arXiv 2604.10658** (independent
|
||||
convergence on our governance; lacks the corpus-substrate moat) → added to landscape-register.
|
||||
Docs: `docs/spec-revision-prior-art-synthesis-2026-07-03.md`.
|
||||
|
||||
**THE FULL SPEC REVISION — RULED BY THE JURIST (all of it).** Via steward relay, the jurist ruled:
|
||||
gaps 1-8 (RATIFIED-CONDITION), gap-7 generative principle (RATIFIED), amendment process (RATIFIED:
|
||||
RFC-supersession + change-class "does this change what any gate accepts" + semver + PENDING/REVIEWED),
|
||||
the three questions (A page-markers, B tier-2 scan = redundant human verification + dinglehopper, C
|
||||
migration-window = `re-verify-pending` status + `verified_under:<semver>` + N=2 provisional + FIX-class
|
||||
extension), and §III citation posture. The UNIFYING INSIGHT: gaps 2+3+item-8 collapse to ONE
|
||||
addressing substrate (apparatus anchors = citation = PROV "original") — design once. Rulings preserved:
|
||||
`docs/spec-revision-RULING-2026-07-03.md` + the gap-6/7 + three-questions + §III rulings (steward relayed).
|
||||
|
||||
**THE LOEB SUBSTRATE REFRAME (the steward's instinct, decisive).** I told the jurist 74% of the corpus
|
||||
(Loeb) was anchorless → they ruled a normative §III extension on it. The steward said "look at the DSL
|
||||
directly." It INVERTED the premise: the canonical anchors (Plato Stephanus `St. II p. 153`, Homer
|
||||
book+line, Aristotle Bekker) are ALL in `loeb.dsl`, FLATTENED by `extract_loeb_dsl.py:47`
|
||||
(`KNOWN_TAG.sub("")` strips the `[c dimgray]` semantic layer). A per-genre anchor audit (agent, re-verified
|
||||
by me against source) confirmed: **>99% of ~2,041 works carry a recoverable anchor**; residual is
|
||||
single-digit genuine fragments; **35,261 `%` index lines carrying book/letter/fragment divisions are
|
||||
discarded**. Disambiguation map captured: `docs/loeb-anchor-audit-and-disambiguation-map-2026-07-03.md`.
|
||||
Jurist re-ruled §III (`docs/chamber-SIII-reruling-JURIST-BRIEF-2026-07-03.md` → ruling): the
|
||||
structured-but-unanchored extension is VOID-as-ratified-for-74% (dormant contingency only), enrichment
|
||||
trigger RETIRED, **§III cites the Loeb by RECOVERED canonical anchor (CTS edition-as-identity)** — work-level
|
||||
fallback only for genuine fragments. Per-genre table NOT ratified (engineering, review post-extraction).
|
||||
Plus a STANDING PROCESS NOTE (jurist): any corpus-population claim feeding doctrine must be sourced from
|
||||
the DSL directly, not a derived extract.
|
||||
|
||||
**Full-run scope decided.** Current Loeb (952) = TIER 1 of `_curation/loeb-canon-proposal-2026-06-15.md`.
|
||||
Since we rebuild the extractor anyway → run the FULL DSL (all 2,041 works / Tier 3). Tier-3's "waits on
|
||||
dilution" is discharged by gap-6's FENCE (corpus-completeness ≠ engine-scope).
|
||||
|
||||
**Tools installed** (steward-authorized): wordfreq → both venvs (closed normalize_ocr's silent OCR-guard
|
||||
degradation); jsonschema → system (--user --break-system-packages, for the validator). dinglehopper
|
||||
kept close (tier-2). Everything else present (pandoc/ebook-convert/docling/rg/jq/node). NB: this machine
|
||||
has **GNU coreutils `stat`** (not BSD) — use `date -r`/python for mtimes, `rg` not `grep -r` for big searches.
|
||||
|
||||
## PRESENT — mood / the arc
|
||||
|
||||
The emotional shape: yesterday ended at "mandelbrot set, whack-a-mole, clutching sand, I tremble at making
|
||||
the engine work." Today is the **inverse** — a marked, finite, ordered plan (ground → instruments → corpus
|
||||
→ fence), the whole doctrine ratified, and the Loeb tier upgraded from "74% impoverished, cite-by-quote"
|
||||
to "richly citable by Stephanus/Bekker/book+line, recoverable." The despair became a siege plan.
|
||||
|
||||
**The load-bearing return of the session (jurist-elevated to standing practice):** I trusted a
|
||||
derived-artifact/regex over the SOURCE **three times** — the break-count (frontmatter fences + bullet
|
||||
lists as breaks), the heading-count (title-as-structure, then line-numbers-as-divisions), and the
|
||||
"anchors absent" (searched "153a" when the DSL uses "St. II p. 153"). The steward twice sent me to the
|
||||
substrate and was right both times; the jurist turned it into a governance rule (source-not-extract for
|
||||
any claim feeding doctrine). This is MY drift-pattern of the session and it must become a Symmetria flag +
|
||||
verification-ladder entry + feedback memory.
|
||||
|
||||
Other recalibrations: I called the §III change a "narrowing"; the jurist sharpened it to "VOID with a
|
||||
dormant residual" — a principle ratified to govern 74% that now governs nothing shouldn't stand as active
|
||||
doctrine (CLAUDE.md: flag a maxim gone decorative). I also owned a citation-integrity miss (cited arXiv
|
||||
2605.24229 for a claim it didn't support; let a sub-agent's blanket "verified" ride) → per-claim citation
|
||||
checking discipline.
|
||||
|
||||
Governance discipline held well: surfaced findings for jurist/steward rulings rather than acting;
|
||||
delegated the audits with anti-pattern-drift framing; verified load-bearing agent claims against source
|
||||
myself before relaying.
|
||||
|
||||
## FUTURE — what is pulling
|
||||
|
||||
**PULLING THREAD (singular): Phase 2 — draft chamber-library-specification.md v2.0 (the superseding,
|
||||
semver-MAJOR version) + build the generative validator.** The doctrine is fully ruled; now MARK THE
|
||||
GROUND. This is dwell-work on the constitution — we deliberately wrapped to give it a fresh, focused start
|
||||
rather than rush it into a marathon tail.
|
||||
|
||||
**ACTIONABLE RESUMPTION POINT (as of wrap — re-judge against what changed):**
|
||||
- **FIRST ACT: read the current `chamber-library/docs/chamber-library-specification.md` end-to-end.**
|
||||
I worked from audits all day, NOT the spec text — and the jurist's own process note (source-not-derived)
|
||||
applies to the spec too: draft against the real text, not my model of it. This is non-negotiable and is
|
||||
itself the first honoring of the day's hardest lesson.
|
||||
- Then draft v2.0 in this order (the substrate first): **§III** = the CTS-URN addressing spine (edition-as-
|
||||
identity + logical passage; the per-genre locator registry as LIVING declared-data, not frozen doctrine)
|
||||
→ **§II** (sectionless+attestor field, break-form `---`, Loeb page-markers route off-canon via the
|
||||
registry) → **§V** (apparatus-integrity gated + anchored on §III; PROV conversion-record mandatory on
|
||||
every path; two gate-functions named: content-grounding vs process-integrity) → **§VII** (declared-gate-
|
||||
list IS the gate; body_word_conservation tracked-unimplemented) → **§IX** (coverage-ledger + freshness
|
||||
invariant + evidence-tier column) → **new §Tiering&Fence** (converted/verified/fenced; promotion criterion:
|
||||
deterministic 93% / scan-50 / fenced-out; V-SUSPECT & V-NONE distinct) → **new §Governance&Amendment**
|
||||
(generative principle + RFC-supersession/change-class/semver + migration-window). Mark HELD: (a)
|
||||
in-band-vs-sidecar (gate reads record NOW; storage form open), (b) markdown-vs-TEI-XML (trigger = if (a)
|
||||
resolves to full in-band). Mark the structured-but-unanchored extension DORMANT (labeled inapplicable to
|
||||
current corpus). Mark OPEN: per-genre table (engineering), tier-2 numeric bar (calibration).
|
||||
- v2.0 is a PROPOSAL per the amendment process: I (champion) draft → jurist editor-gate → steward ratifies →
|
||||
semver v2.0 supersedes v1 (Obsoletes/Updates). Then build the generative validator (graduation-spec.yaml
|
||||
carries the convention-data incl. the per-genre registry; gates become thin consumers; JSON-Schema for
|
||||
structured layers). THEN Phase 3 (rebuild extract_loeb_dsl.py to the disambiguation map, run the full DSL).
|
||||
- All conditions are captured in PENDING-46. The one-screen plan: ground → instruments → corpus → fence
|
||||
(④ waves come AFTER ② — waves-first bakes tool-shaped conventions = the redo we refuse).
|
||||
|
||||
**Other open horizons, ranked:**
|
||||
- The generative validator (part of ②) — the JSON-Schema/PROV/thin-consumer-gates build.
|
||||
- Phase 3: the full Loeb re-extraction (2,041 works) — depends on ② + the disambiguation map (captured).
|
||||
- The remediation waves (④): A frontmatter / B1 re-clean / B2 structure-recovery / C apparatus — gated by the fixed tools + validator.
|
||||
- Gap-6 promotion + fence (⑤): deterministic tier (93%, word-identity re-convert), scan tier (50, dinglehopper+calibration), fenced-out.
|
||||
- Paper debt: REVIEWED entries owed for the whole PENDING-46 arc (steward writes). dinglehopper install (tier-2).
|
||||
- The 3 pre-existing untracked leftovers in chamber-library (`_scratch/`, `nature-of-order-alexander.md`) — NOT mine.
|
||||
|
||||
**PAUSE STATEMENT:** Away now at a high, clear point — the entire doctrine phase closed in one long day,
|
||||
the despair of yesterday turned into a marked plan. What I most want to find still pulling on return: **the
|
||||
drafting of v2.0** — that the next hand opens the actual spec text FIRST (not my model of it), then drafts
|
||||
§III the addressing spine, giving the constitution the dwell it asks for. Do NOT rush it; do NOT skip the
|
||||
end-to-end read (it's the day's hardest lesson honored).
|
||||
|
||||
**LITERAL QUESTION for next-Claude:** *When you read the current `chamber-library-specification.md`
|
||||
end-to-end (which I never did today — I worked from the audits), does the real text hold any structure,
|
||||
constraint, or already-settled decision that the day's audits missed or would collide with — before v2.0 is
|
||||
drafted against my model of the spec rather than the spec itself?* (The source-not-derived discipline, turned
|
||||
on the spec.)
|
||||
|
||||
**State:** chamber-library `main` — Wave 0 (f9cbb8e) + docs (1c88650) committed, LOCAL/unpushed; 3 new
|
||||
session docs untracked (jurist briefs + audit — commit at wrap). CM-AI landscape-register.md modified
|
||||
(Seck add). dotfiles: PENDING.md + session-ledger untracked (commit+push at wrap). All doctrine ratified;
|
||||
Phase 2 unblocked.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: session-ledger-2026-07-03
|
||||
description: "Practice-of-return ledger maintained by /symmetria — returns, open horizons, recalibrations, authorization moves, sub-agent dialogues, bypasses."
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: feedback
|
||||
originSessionId: 5b7ce298-123b-447a-9b86-2442c30bec1f
|
||||
---
|
||||
|
||||
# Session Ledger — 2026-07-03
|
||||
|
||||
## Returns
|
||||
- 2026-07-03 — check before writing `audit_corpus.py`. Three commitments logged: (1) the `verify_graduation.py` refactor touches a GOVERNED gate (PENDING-44) — must be behavior-preserving + PROVEN byte-identical stdout on a sample before trusting; subprocess fallback if proof fails. (2) Scope: conventions+apparatus on the 332 non-Loeb only (Loeb 952 = separate DSL tier; the convention gate would false-FAIL them); text-health on all 1284. (3) Apparatus check CLASSIFIES + emits evidence counts (none/paired-ok/orphaned/stranded-suspect), never adjudicates; validate the classifier against Sennett (→paired-ok) + a real editorial-note file (→not-stranded) before trusting — per census-through-a-pattern flag. Data already surfaced: 8 files with live `[^]:` machinery vs 68 with a Notes heading ⇒ ~60 stranded-suspects.
|
||||
|
||||
- 2026-07-03 — apparatus classifier blind-spot CAUGHT by validating against real files (not just synthetic self-test): Taruskin/Beethoven had 89/131 `[^x]` refs with ZERO defs (all dangling) yet were classified `notes-editorial` (benign) because an editorial-style heading matched and the `orphaned` branch only fired when defs>0. Fix: refs-present-without-defs → `orphaned`, caught before the heading fallthrough. Without the real-file validation the map would have marked ~dozens of damaged files clean. The census-through-a-pattern flag paid off exactly as predicted.
|
||||
- 2026-07-03 — GOVERNED-gate touch (PENDING-44): refactored `verify_graduation.py` to expose `collect_checks()`/`resolve_spec_path()` so the auditor and the gate share one convention source. Proven byte-identical stdout + exit on PASS(Sennett) and FAIL(Plato) samples vs git HEAD before trusting. Behavior-preserving [FIX]; paper note owed alongside PENDING-44/45.
|
||||
|
||||
## Open horizons
|
||||
- 2026-07-03 — MAP BUILT. `audit_corpus.py` + `corpus-quality-ledger.tsv` (1284 rows) + interpreted `corpus-quality-map-2026-07-03.md` with proposed waves. Distribution: Loeb 930/952 clean (corpus healthier in aggregate than the 332-view); non-Loeb 8 clean, 324 A-convention (mechanical), 110 B-textdamage, 70 C-apparatus (NEW axis). Wave A (frontmatter migration, ~99% mechanical, clears 170 A-only to green) recommended as first countdown. AWAITING steward authorization to run a wave — do not begin unilaterally.
|
||||
- 2026-07-03T00:00 — Woke into the confirmed pulling thread: build `scripts/audit_corpus.py` + apparatus check → full quality ledger (`_curation/corpus-quality-ledger.tsv`) of ~1,284 works. Verified the map does NOT yet exist. Standing return to hold: **do not resume graduating reactively** — the mole-by-mole reflex is the pattern that produced last session's distress; measure the whole first.
|
||||
|
||||
- 2026-07-03 — steward challenged the framing (correctly): are the TOOLS spec-compliant before we run any wave? "The tool IS the wave — a stale cleaner run at scale re-corrupts 300 files." Confirmed gaps from reading spec+runbook+tool-log: (1) §VII gate-fix is [OPEN] — verify_conversion heading_density FALSE-FAILS legitimately sectionless works (La Chute), so the map's 59 "B2 structure-recovery" count is CONTAMINATED (conflates damage with sectionless-by-nature). (2) NO gate checks apparatus/footnote-pairing (§V keep_notes) — why 70 damaged files graduated. (3) NO per-language dict-validity check — passed Levi's capacitv/Alanichaean char-substitution. (4) 13/15 mutating/gate tools are STANDALONE (don't reference the governed spec → nothing prevents drift). (5) graduate_to_canonical missing the 2026-07-02 source-archiving wiring (TODO). ⇒ launching fleet compliance audit (3 agents) before authorizing Wave A.
|
||||
|
||||
- 2026-07-03 — MAP CORRECTION (verified): of 59 heading_density fails, 46 are the §VII sectionless FALSE-FAIL (spec-valid — Plato Republic, Sontag, Berger's About Looking w/25 breaks) and only 13 are genuine structure-loss (some of those, e.g. Rilke Malte, may also be sectionless). B2 was inflated ~4.5×. Running B2 blind would have FORCED headings onto ~46 sectionless works = §II corruption. The gate needs the §VII fix before B2 is even definable. Steward's tools-first instinct vindicated.
|
||||
- 2026-07-03 — RETIRE candidates (steward asked: complete kit, nothing useless). Full scripts/ census by ref-count: ref=0 → clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings; + clean_pdf_running_heads (hardcoded "Auerbach Mimesis" — book-specific); + clean_ocrmac_pagination (superseded by normalize_ocr per runbook). Candidates only — agents confirming redundant-vs-unique before retiring (don't assert-absence-from-grep).
|
||||
|
||||
## Confidence to recalibrate
|
||||
- 2026-07-03 — RECALIBRATION (honest): my morning map-correction "46 sectionless-valid / 13 loss" was WRONG — the quick break-regex counted frontmatter `---` fences AND markdown bullet lists (`^\*\s*\*\s*\*` spanning newlines) as thematic breaks. Fell into census-through-a-pattern on my OWN correction (the Symmetria §3 flag, same day I relied on it for the classifier). TRUE picture: 9 files have real markdown thematic breaks (auto-rescued by §VII fix); ~50 need per-file judgment split into (a) genuinely-sectionless→declare sectionless:true, (b) `• • •`/`**•**`-style separators = a §II.2 break-normalization defect (literal glyph-runs render as prose = no represented structure). The §VII gate fix is CORRECT (asserts "structure represented in markdown"); only my earlier COUNT was contaminated. Lesson re-confirmed: anchor break-detection to the SPEC's break form (`---`), verify the regex against real files, don't span newlines.
|
||||
|
||||
|
||||
## Authorization moves
|
||||
|
||||
## Sub-agent dialogues
|
||||
- 2026-07-03 — Agent C (structure+graduation). audit-agent: PASS (0.95 conf, read every line, greps confirm absence, uncomfortable-not-convenient finding, distinguishes verified/inferred). VERIFIED MYSELF: graduate_to_canonical calls verify_graduation (L63) NOT verify_conversion (0 refs) — only audit_cruft cruft==0 (1 of 5 health checks). git mv no git-add (L133). NO source-archiving (manifest path edits only). graduation-spec.yaml L74-77 DECLARES verify_conversion+verify_graduation+body_word_conservation required → tool violates its own spec. ⇒ the graduation GATE is half-missing: cruft-free-but-damaged files graduate. + verify_graduation itself under-enforces gaps (hardcoded 3 of 6 placeholder patterns). Structure tools healthy (insert_chapter_headings exemplary fail-loud; structure_from_ncx & repair NOT redundant — keep both).
|
||||
- 2026-07-03 — Agent A (cleaners). audit-agent: PASS (0.85-0.95, negative-grep rigor, honest empirical-gap caveat). GLOBAL GOOD: NO prophylactic char-normalization in any of 5 (§V Tier-2 honored — the key wave reassurance). strip_cruft (PRIMARY): prose-guard is EXTERNAL/advisory, RESIDUAL>0 prints "don't graduate" then WRITES anyway → port internal word-guard + block-on-residual. clean_pandoc_html_residue: best-guarded, wave-safe. clear_safe_html_residue: orphaned/redundant except <nav> → fold+retire. clean_calibre_artifacts: orphaned + real PASS-BUT-FALSELY (strips <a> footnote links→bare numbers→passes guard; predates keep_notes) → retire or add footnote-preflight. clean_pdf_running_heads: hardwired-Auerbach, no guard/backup, blanket ^\d{1,4}$ strip → NOT wave-safe, rename to oneshot + fix runbook L103.
|
||||
|
||||
## Meta-findings (the class behind the instances — steward asked "what's hiding in plain sight")
|
||||
- 2026-07-03 — (1) ONE failure shape fleet-wide: gates enumerate known-bad SIGNATURES not the INVARIANT (normalize_ocr count-not-identity; verify_conversion cruft-signatures→Taylor <sup>; verify_graduation 3/6 patterns; audit_cruft signature-list; strip_cruft advisory-residual). Wave-0 fixes must all be assert-the-invariant shape. (2) The fleet has NO self-test — every bug found by reading; only audit_corpus (written today) has --validate. ⇒ build a FIXTURE HARNESS (known-good+known-bad per defect class), do each fix test-first. (3) The gate is a discipline not a door — graduate should be the single fail-closed chokepoint. (4) Checks need ONE home shared by gate+map (apparatus lives only in map now → drift); bigger: build_catalogue could carry live gate-status = standing health ledger (§IX coverage). (5) §V Tier-2 conversion-record written only on OCR path, not EPUB/cleaner path — wider §V gap. Ideas 4-ledger + 5-record → follow-on [HARDENING], not Wave 0.
|
||||
- 2026-07-03 — STEWARD AUTHORIZED all 8 Wave-0 blockers (after this reflection). Method: fixture harness first, then 8 fixes test-first, invariant-not-signature, gate-as-single-door, shared checks. Tool-fleet-compliance ledger written: _curation/tool-fleet-compliance-2026-07-03.md.
|
||||
|
||||
## Wave-0 progress (test-first, harness = scripts/test_tools.py)
|
||||
- 2026-07-03 — BUILT scripts/test_tools.py (the fleet self-test the tools never had; no pytest dep, matches --validate idiom). Each fix pinned by a fixture asserting the INVARIANT it restored.
|
||||
- #5 DONE — verify_conversion §VII sectionless: heading_density passes when 0 headings + real markdown thematic break (---/***) OR sectionless:true; only FLAGS glyph-run separators (• • •) which are a §II.2 break-normalization defect. 3 fixtures green. Recalibrated the map: 9 real-break auto-rescued, ~50 need per-file judgment (not the wrong "46").
|
||||
- #1 DONE — graduate_to_canonical.gate_candidate(): the single door now runs BOTH verify_conversion (health) AND verify_graduation (conventions), imported (no more stdout-parsing subprocess). cruft-free-but-damaged + non-conformant both GATED; healthy+conformant passes. 3 fixtures green. Closes the hole that let 110 damaged files graduate.
|
||||
- #6 DONE — verify_graduation gap check drives from the FULL spec placeholder_patterns list but only inside a gap FRAME (HTML comment / [bracket]); prose false-positives avoided; STEWARD-RULED exempt. 5 fixtures green; green-set regression still passes.
|
||||
- #2 DONE (load-bearing) — normalize_ocr.verbatim_guard now ALWAYS aligns R↔C (merge=concat / doubling=drop-head, per-type counts verified); no more identity-skip when merged_count>0. Catches word-swap-with-balancing-count + unrecorded-join. 4 fixtures green. Signature: (raw,canon,heads,n_merged,n_doubled). Follow-on (noted, not blocker): merge_flag hard-stop policy + wordfreq-venv degradation.
|
||||
- #3 DONE — strip_cruft: in-tool prose-safety guard (never bypassable, delegates verify_conversion.prose_delta) + block-on-residual (--allow-residual override); guards run BEFORE backup (refusal = no side effect). 3 fixtures green.
|
||||
- #4 DONE — clean_epub_residue: FOOTNOTE_RESIDUAL tripwire raises FootnoteResidual before the TOC-unwrap can silently destroy an uncovered footnote family (#filepos/bare-fn); main aborts loud. Real TOC cross-ref doesn't false-trip. 2 fixtures green.
|
||||
- #7 DONE — graduate: git add before git mv (+ plain-move fallback) so untracked batches never silently graduate 0; honest failure count; loud SOURCE-ARCHIVING-NOT-WIRED warning (2026-07-02 directive is a noted follow-on feature-build, not a wave blocker).
|
||||
- #8 DONE — retirements: archived 6 orphaned/dead tools → scripts/archive/ (clean_ocrmac_pagination, clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings, clean_pdf_running_heads[Mimesis one-shot, job done]). FOLDED <nav>-block removal into strip_cruft first (capability preserved, fixture-pinned). Fixed runbook's dangerous generic clean_pdf_running_heads reference. No live tool imports an archived one.
|
||||
- ALL 8 WAVE-0 BLOCKERS DONE. Harness scripts/test_tools.py = 21 assertions across 6 tools. Follow-on [HARDENING] (noted, not blockers): source-archiving auto-wire; merge_flag hard-stop + wordfreq-venv pin; §V conversion-record on EPUB/cleaner path; build_catalogue-as-standing-health-ledger (meta-idea 4). NOT committed yet.
|
||||
|
||||
## Prior-art scans (steward: "incredible we're the only ones"; generative-from-spec principle)
|
||||
- 2026-07-03 — Scholarly-encoding scan (audit-agent PASS 0.95, primary sources read). VERDICT: we are NOT first — ~70% of our spec re-derives TEI + CTS + Standard Ebooks. ADOPT: (1) TEI `<choice>` never-destroy-the-original (orig/reg,sic/corr) — vs our replace-and-sidecar; question for gap-3. (2) TEI ch.13 apparatus `<app>/<lem>/<rdg>` — anchor to stable LOCATION not footnote NUMBER + verify resolution → hardens gap-2. (3) CTS URN — structural addressing + EDITION-AS-IDENTITY → what §III should become. (4) Standard Ebooks `se lint` gate (validates our gate arch) + `[Editorial]`-isolated-commit provenance + per-defect detector scripts (audit_corpus template). (5) Distributed Proofreaders: redundant verification AGAINST THE SCAN (single pass can't ground verbatim); HathiTrust: no ground-truth at scale → quality-tiers (validates map+waves). GENUINELY OURS (keep): checksum byte-integrity (TEI/CTS lack it, ~0.75), LLM-literal-citation constraint, corpus-as-trust-substrate governance. TEI ODD/Roma = spec-generates-validator = the steward's generative principle, already a standard → likely the model. 2 scans still out (tooling/spec→validator, bounded-corpora-for-AI).
|
||||
|
||||
- 2026-07-03 — Tooling/spec→validator scan (audit-agent PASS 0.85, primary sources; the generative-principle answer). TEI ODD/Roma = "One Document Does it All": one spec compiles to BOTH validator AND docs, can't drift = purest generative-from-spec, BUT XML-locked (full adoption = corpus→TEI-XML). VERDICT: realize the principle NOW without full TEI — W3C PROV (PROV-O) = standard vocab for gap-3 conversion-record (adopt, don't invent schema); JSON Schema = free spec→validator for structured layers (.meta.json sidecars, content-types.yml, class enum); PANDOC LUA FILTERS = reimplement cruft/footnote recovery on the parsed AST (structure KNOWN not regex-guessed) → dissolves clean_epub_residue's whole failure family at root; dinglehopper (OCR-D) for GT-diff; ftfy for encoding. GENUINELY OURS (no equivalent anywhere): prose-word-multiset guard (field optimizes plausibility→silent mutation; ours forbids alteration), verbatim-span-in-bounded-source check (RAG does semantic NLI not lexical), checksum integrity. HELD-OPEN FORK [PROPOSAL]: markdown+JSON-Schema now (proportionate) vs TEI-XML+ODD if engine fidelity outgrows markdown — name, don't resolve. 1 scan out (bounded-corpora-for-AI). Then: consolidated synthesis → jurist spec-revision brief (PENDING-46).
|
||||
|
||||
## Bypasses
|
||||
@@ -288,3 +288,13 @@ The single place proposed skills live so they don't evaporate between sessions.
|
||||
| **Symmetria §3 flag: re-derived-instructions-instead-of-citing-the-governed-doc** | Symmetria §3 flag | Wrote agents hand-made instructions (and invented fields) instead of pointing them at `conversion-runbook.yaml`/the spec — the direct cause of the graduation divergence. Kin to `reinvented-governed-tooling-without-checking-it-exists`, at the *instruction* layer. Antidote: cite the governed doc; if it's silent, that's a runbook gap to close, not a prompt to improvise into. | Symmetria §3 | **PROPOSED** |
|
||||
|
||||
*(Four. The load-bearing two: BUILD `/graduate-chamber-source` now [it's the throughput multiplier for the whole Making batch], and record the graduation-spec+validator+gate as a named governed instrument [the steward's own idea, proven]. The two §3 flags are the session's real returns — both steward-caught, both class-fixed in the spec/gate. Not manufactured.)*
|
||||
|
||||
### Harvest 2026-07-03 (doctrine phase — Wave 0, full spec-revision rulings, Loeb substrate reframe)
|
||||
|
||||
| Element | Kind | One-line | Where it lands | Status |
|
||||
|---|---|---|---|---|
|
||||
| **Symmetria §3 flag: claim-from-derived-artifact-when-the-source-is-checkable** | Symmetria §3 flag | **The load-bearing harvest — jurist-elevated to STANDING PRACTICE.** I trusted a derived artifact / a regex-over-derived-text over the AUTHORITATIVE SOURCE **three times in one session**: the break-count (frontmatter fences + bullet-lists counted as thematic breaks), the heading-count (title-as-structure, then line-numbers-as-divisions, Plutarch-chapters missed), and "anchors absent" (searched Stephanus form `153a` when the DSL uses `St. II p. 153`). The steward twice sent me to the substrate (`loeb.dsl`) and was right both times; the jurist ruled it a governance rule: *any corpus-population claim feeding spec doctrine must be sourced from the source directly, or explicitly flagged extract-derived+provisional, before it does normative work.* Same discipline gap-2/gap-3 enforce on the corpus, turned on how the corpus is AUDITED. Antidote: verify against the authoritative source, never the derived form, when the source is checkable — and validate a matcher against a known positive AND negative before trusting its count (kin to census-through-a-pattern, generalized to source-vs-derived). | Symmetria §3 + `reference-verification-ladder.md` (a named "source-not-derived for doctrine-feeding claims" entry) + a feedback memory | **PROPOSED (strongly earned — 3× + jurist-ratified as standing practice)** |
|
||||
| **`/spec-amendment`** (the RFC-supersession amendment process) | create (later) | The now-RATIFIED chamber amendment process as a codified discipline: normative spec change = a superseding version (Obsoletes/Updates, never in-place edit) + change-class test ("does this change what any gate accepts?" → FIX vs PROPOSAL) + semver-MAJOR re-verifies all consumers + PENDING/REVIEWED record. Recurs every spec amendment. **Defer** until v2.0 is drafted (the process's first real exercise IS Phase 2 — build the skill from that empirical run, don't pre-author). | new skill | PROPOSED (build-after-first-use, i.e. after Phase 2) |
|
||||
| **per-claim citation verification (don't let a sub-agent's blanket "verified" ride)** | feedback memory or verification-ladder note | Caught by the jurist: I cited arXiv 2605.24229 for a claim it didn't support, having let a sub-agent's aggregate "4/4 papers verified" stand without per-citation checking. Verify each citation against the SPECIFIC claim it carries; never let an agent's blanket verification substitute for per-claim checking. | feedback memory / ladder | PROPOSED |
|
||||
|
||||
*(Three, all earned. The first is the session's real return — a drift I hit three times and the jurist turned into a standing rule; it should become a Symmetria flag AND a ladder entry AND a feedback memory. The second is a genuine future skill but correctly deferred to after its first real use. The third is a concrete citation-integrity guardrail from a real miss. None manufactured.)*
|
||||
|
||||
Reference in New Issue
Block a user