session 2026-07-12: footnote arc closed + end-to-end verify + program open-work register (PENDING-55 Q1a-c/Q2, retroactive sweep); pulling into A1 verbatim gate

This commit is contained in:
David F Glidden
2026-07-12 08:09:44 +02:00
parent 5ec438ce72
commit 043a4b1836
7 changed files with 173 additions and 1 deletions
+40
View File
@@ -1427,3 +1427,43 @@ Disposition (verbatim capture + executor action-list: `chamber-library/docs/libr
- **§5 wording cleaned** in the brief (cross-file *mechanism* is covered/proven — relocation on Mauss/Polastron; the Orwell/Quixote *exact shape* isn't a separate fixture yet — "mechanism covers it, not isolated"). No self-contradiction.
- **§7 Jung provenance — fact-check delivered for steward+jurist (open, non-blocking):** the corpus does BOTH — **per-volume precedent EXISTS** (Alexander *Nature of Order* 4 files · Lacroux *Orthotypographie* 2 · Habermas vol-1 · Burney vol-1 · Camus *Œuvres I*) and single-megafile (Xenophon/Muir/Zhuangzi/Rumi/Levi/Blake/Shakespeare complete). Operative distinction ≈ per-volume citation identity; Jung CW (cited by volume) fits the per-volume side, supporting the jurist's lean. Steward+jurist decide; governs re-converting the legacy Jung `.md` (+ likely Donne).
- **Not yet done (post-authorization work):** recognizer coverage of the remaining census families (one-directional / container-mirror); semantic per-volume boundary detection (if per-volume chosen); build-in under fleet discipline (`test_tools` fixtures, spec anchoring); re-convert the legacy mega-books. Steward assigns a PENDING-number-consistent build authorization.
---
## PENDING-55 — Re-scope the REVIEWED-54b remaining footnote work on an accurate census (new nested-block family; (c) is a census artifact)
**Date:** 2026-07-11
**Tag:** [PROPOSAL]
**Summary:** A new read-only diagnostic (`scripts/audit_footnotes.py`, recognizer-truth for coverage) replaced the five throwaway probes and produced a census materially more accurate than the one REVIEWED-54b was ruled on. It changes the *un-built* remainder of that ruling in three ways, so — per steward direction (2026-07-11) — building is **held** pending a jurist re-scope: (1) **(c) "single-letter marker" is a census artifact** — no single-`[A-Za-z]` book exists; nagarjuna = inner-anchor digit → (b), plato = block-id digit → already COVERED, and **donne is a whole-word/phrase lemma-marker book** (~2150 notes, exact/block, ~60% exact reciprocity) that is *broader* than the ruling and removes the marker-shape defense-in-depth; (2) a **NEW family the prior census never saw — nested-block** (~354 notes: les-fleurs / jaccottet / polastron; note-id on an outer `<div>`, back-link nested in an inner `<p>`; exact reciprocity, low risk); (3) the honest **blind zone** (44 `REFS-BUT-NO-RECIPROCITY` books) names the real one-directional/container notes (arendt / montaigne / doniger / métamorphoses) among genuine TOC/index non-notes.
**Full brief (for the jurist):** `chamber-library/docs/footnote-family-census-FOR-JURIST-2026-07-11.md`. **Evidence artifact:** `docs/footnote-family-census-2026-07-11.txt`.
**Change-class:** [PROPOSAL] — it re-opens what the REVIEWED-54b conversion-pipeline ruling authorizes building, and bears on §V's production chain (same test as PENDING-54). The diagnostic tool itself is read-only (no gate/spec/verbatim change) and is already built + tested (fleet 83→86) under the steward's "build the durable tool" direction.
**REVIEWED-54b part 1 unaffected:** (a) block-id + (d) URL-decode are landed + verbatim-proven (`ef0cee0`). This item concerns only the un-built (b)/(c) + the newly-surfaced families.
**Open questions (brief §6):** (1) authorize donne's word/phrase-marker widening under strict exact reciprocity + n=3 spot-check, or hold donne to the regex fallback and drop (c)? (2) nested-block = FIX (extend the block-note pass to search block *descendants*) or PROPOSAL? (3) blind-zone one-directional/container notes = new families or confirmed honest-refuse? (4) confirm (b) inner-anchor proceeds under its existing FIX ruling; (5) proof standard unchanged (per-pair `(dest-file,id)` guard + `_prove.py` verbatim on real books); (6) optional analyzer-precision FIX — expose paired-ref positions from `process()` for exact per-ref attribution.
**Files affected (on authorization):** the recognizer `scripts/inject_epub_footnotes.py` + `test_tools` fixtures + `graduation-spec.yaml` note, per whichever families are authorized. None touched this session (building held).
**Awaiting:** Steward + jurist re-scope ruling. Building resumes only after.
**RULED 2026-07-11 (jurist; REVIEWED-55). The ruling CORRECTS the jurist's own prior REVIEWED-54b (c) sub-ruling** — reasoned correctly from data that was wrong; the analyzer stress-test surfaced it one layer up, in an issued ruling, before donne was converted on the false premise. Full ruling: `docs/footnote-family-census-JURIST-RULING-2026-07-11.md` (relayed). Disposition:
- **(c) letter-marker — WITHDRAWN in full** (incl. the n=3 spot-check condition — no subject left). No letter-marker family exists: nagarjuna = digit/inner-anchor → **(b)**; plato = digit/block-id → already covered by (a).
- **donne (word/phrase-marker) — DECLINED for this arc.** Whole-word/phrase markers are the *specific* failure condition-4 exists to prevent (indistinguishable from prose by shape; round-trip would carry all the safety weight) — a different risk, not a bigger one, so NOT (c)'s natural extension. The measured 60% exact-reciprocity seals it regardless (partial, self-inconsistent conversion vs a uniform working fallback). **Hold to the regex-cleaner path;** a fresh PROPOSAL if ever pursued (not a rider). The ~40% non-exact = probable source-side id typos → **anchor-hygiene note (logged, not ruled), alongside Sennett `cra0001002`** → `_curation/anchor-hygiene-findings.md`.
- **nested-block (~354) — FIX, same arc.** Widen the block-note back-link search from immediate children to **descendants** — widens *where* the same unchanged match is looked for, doesn't loosen *what* matches; same verification/risk as (a). Variants: don-quixote `<body>`-id = genuine false candidate (TOC≠footnote, leave out); obrist native `<aside epub:type=footnote>` = **confirm empirically through stock pandoc** before treating as covered OR gap.
- **blind-zone one-directional (arendt/montaigne/doniger/les-métamorphoses) — confirmed honest-refuse.** No round-trip → nothing for the guard to verify → out of scope *by architecture*, not omission (machine-converting them = PROPOSAL-class, different first principles). Regex-cleaner fallback per REVIEWED-54 §5.
- **(b) inner-anchor — FIX stands, scope corrected to INCLUDE nagarjuna.** of-grammatology (double inner anchor, symbol) noted adjacent, NOT folded in — its own later look.
- **proof standard — unchanged/confirmed** (per-pair `(dest-file,id)` guard + verbatim-clean real-book proof, +0/−0 — it's what makes these FIX).
- **analyzer precision fix — AUTHORIZED, FIX** (expose `process()`'s paired-ref positions; pure additive instrumentation; build soon — would have caught the misclassification pre-ruling).
**Build UN-HELD for:** (b) inner-anchor (incl. nagarjuna) · nested-block · analyzer-precision fix — all FIX under the arc's proof standard. **Awaiting only:** steward placement of **REVIEWED-55** (executor can't write REVIEWED.md — Constraint #1; paste-ready draft in the session ledger).
**BUILT 2026-07-11 (chamber `b65816a`, `3831878`):**
- **Analyzer precision fix** (FIX): `inject_epub_footnotes.process()` exposes `paired_refs`; `audit_footnotes` attributes coverage exactly per-ref. Regression-proven.
- **(b) inner-anchor** (FIX): PROVEN +0/−0 on kuhn/mbembe/nature-of-things (916 notes); i-ching honest-refused (wrapper-div nesting); nagarjuna exposed a **PASS-BUT-FALSELY** (its index cites the notes → converting them perturbs the index; source-word guard passed while pandoc dropped body words).
- **⚠ NEW SAFETY MECHANISM — end-to-end verify (steward-directed, `3831878`): bears on the §V proof standard → JURIST NOD OWED.** After writing, the pre-processor runs stock pandoc on the original + injected EPUB and compares body-word multisets; ANY non-marker change REFUSES + deletes the output. Turns the guarantee from *verbatim-safe by construction* → *by verification*; nagarjuna self-refuses; closes the PASS-BUT-FALSELY class; also compensates the unenforced `body_word_conservation` gate for footnote conversions. Steward authorized the build under REVIEWED-55's FIX proof-standard clause; surfaced here for the jurist to ratify the mechanism's place in the §V chain.
- **nested-block — jurist FIX-assumption OVERTURNED by proof (⚠ needs a jurist note):** built the widen (search block descendants for the back-link); it PAIRS the notes but converts DIRTY (les-fleurs +828/−1110, jaccottet marker-merge, polastron +202/−2756) — pandoc won't cleanly lift a `<div class="note">`-wrapped same-file def; same PASS-BUT-FALSELY class as i-ching/nagarjuna. The end-to-end verify catches it (vindicates it). "Widen the search = same as (a)" doesn't hold empirically. **REVERTED to `3831878`** (the recognizer must not pair what it can't convert clean). Recommend nested-block joins the **honest-refuse** set (regex fallback), like i-ching. Second ruling-assumption this arc overturned by the proof (after (c)).
- **obrist native-`<aside>` — empirically a GAP:** stock pandoc emits 0 native footnotes from its `<aside epub:type="footnote">`; 16 notes, low priority → regex fallback (not covered, not a build target now).
- **DONE this arc:** analyzer-precision fix + (b) inner-anchor + end-to-end verify (all `3831878`/`7bf438a`, both remotes). nested-block + obrist → honest-refuse.
- **JURIST RULED 2026-07-12** (`docs/end-to-end-verify-and-nested-block-JURIST-RULING-2026-07-12.md`, relayed & archived):
- **Q1a — end-to-end verify RATIFIED, FIX-class** on a refined test: a new §V-chain component is FIX (not PROPOSAL) when it (i) only ADDS refusals, (ii) never alters text, (iii) falls back to an already-trusted path. This satisfies all three.
- **Q1b — MANDATORY at graduation now** (`--no-verify` forbidden at graduation while `body_word_conservation` is unenforced) **and should PERSIST after that gate lands** (defense-in-depth: catch at conversion-time, attributed to source). **+ RETROACTIVE re-verification REQUIRED** over everything landed under the per-pair-only standard (mirror/split-anchor/block-id/URL-decode) before treating it settled.
- **Q1c — marker-exclusion must be POSITIONAL.** ✅ **DONE** (`e8eeea7`): was shape-only; now subtracts only the exact converted marker tokens (`converted_markers`), superscript-normalized; a dropped prose number is caught though digit-shaped; clean books hold, nagarjuna refuses on real corruption only.
- **Q2 — nested-block FIX RETRACTED, honest-refuse CONFIRMED.** Fresh PROPOSAL required if ever revisited (unwrap-the-div is a different approach). Jurist named its own reasoning gap ("same risk as (a)" was true of matching, silent on rendering) + generalized: **no widen ruled FIX-and-final on structural reasoning alone; structural safety = provisional FIX, end-to-end verbatim proof on real books = final.**
- **Broader flag (jurist):** `body_word_conservation` declared-but-unenforced is the same gap-shape PENDING-52 closed, recurring in a second subsystem → worth renewed attention beyond this arc.
- **DONE:** Q1c positional (`e8eeea7`); Q1a/Q2 archived.
- **Q1b RETROACTIVE — MAJOR finding (`6c9ca43`, evidence `docs/retroactive-verify-2026-07-12.txt`):** 87 covered books · **42 CLEAN · 45 DIRTY** (pass per-pair guard, corrupt under real pandoc). Verified real (5 known-clean → CLEAN; `_prove.py` agrees). 29 catastrophic (montaigne +385457, red-book −7835, pascalian +5910, gadamer/donne/tolkien…; false-pair relocation = nagarjuna at scale) + 16 marker-adjacency (prose intact, markers glued). **CORPUS SAFE: 0 canonical files use the pre-processor — NONE graduated; caught pre-graduation; mandatory verify refuses all 45 → fallback.** Recognizer's reliable coverage ≈42/87. **⚠ NEW DECISION for steward+jurist:** improve the recognizer (tighten over-pairing) vs accept 42-clean + fallback-for-the-rest. Not resolved this session.
- **OWED (steward, tomorrow):** REVIEWED-55 + follow-on placement; wire "mandatory-at-graduation" (graduation flow must not pass `--no-verify`); weigh the recognizer-maturity decision above.