session 2026-07-12: PENDING-56 (canonical-format decision, jurist-ratified) + in-flight sidecar build report + session memory/ledger/REVIEWED-drafts

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jYS1tpR4P4NBxui4FSgfv
2026-07-12 12:15:15 +02:00
parent 043a4b1836
commit 0a424aa6b3
9 changed files with 200 additions and 1 deletions
@@ -1467,3 +1467,14 @@ Disposition (verbatim capture + executor action-list: `chamber-library/docs/libr
 - **DONE:** Q1c positional (`e8eeea7`); Q1a/Q2 archived.
 - **Q1b RETROACTIVE — MAJOR finding (`6c9ca43`, evidence `docs/retroactive-verify-2026-07-12.txt`):** 87 covered books · **42 CLEAN · 45 DIRTY** (pass per-pair guard, corrupt under real pandoc). Verified real (5 known-clean → CLEAN; `_prove.py` agrees). 29 catastrophic (montaigne +385457, red-book −7835, pascalian +5910, gadamer/donne/tolkien…; false-pair relocation = nagarjuna at scale) + 16 marker-adjacency (prose intact, markers glued). **CORPUS SAFE: 0 canonical files use the pre-processor — NONE graduated; caught pre-graduation; mandatory verify refuses all 45 → fallback.** Recognizer's reliable coverage ≈42/87. **⚠ NEW DECISION for steward+jurist:** improve the recognizer (tighten over-pairing) vs accept 42-clean + fallback-for-the-rest. Not resolved this session.
 - **OWED (steward, tomorrow):** REVIEWED-55 + follow-on placement; wire "mandatory-at-graduation" (graduation flow must not pass `--no-verify`); weigh the recognizer-maturity decision above.
+
+## PENDING-56 — Canonical format decision: adopt Docling + complete MD-canonical + TEI-mirroring structural sidecar
+**Date:** 2026-07-12
+**Tag:** [ESCALATE] architectural — canonical format (seed brief §10.3, deferred as steward-pending). **Steward-decided provisionally on the trial evidence; jurist ratification requested (brief relayed).**
+**Summary:** Adopt Docling as the EPUB/PDF/scan conversion front-end (with our verify gates unchanged on top), and *complete the already-chosen* Markdown-canonical + `.meta.json` structural-sidecar architecture — giving the sidecar a TEI-mirroring vocabulary so a future TEI migration stays mechanical.
+**Rationale:** A steward-driven reframe established that the substrate (conversion/format/citation) is a solved, adoptable domain (Docling/GROBID/TEI/DTS) while the composition (voice-attributed verbatim consultation = the engine) stays ours — and that ~20 bespoke conversion tools were reimplementing it. A four-tier Docling trial (steward-run, M4+M1) confirmed: value scales with layout richness — cleans EPUB (0 cruft, was 888 calibre_link/364 fn-ids), rich structure+provenance on PDF (2 798 page-provenanced items), and **crosses the A3 OCR frontier** on a scanned French book (104 footnotes, 664 provenanced) that `ocrmac` cannot. Docling is a *converter, not a verifier* — the trust gates (`verify_body_conservation`, `normalize_ocr` verbatim-guard) stay, converter-agnostic.
+**Options:** **A** — MD-canonical + TEI-mirroring `.meta.json` sidecar (tractable now; completes the chosen half-built direction; human-inspectable; keeps TEI open). **B** — TEI now (native to apparatus A5 / anchors B2 / DTS, but a full TEI toolchain before any corpus work resumes). **C** — split (TEI for Loeb, MD+sidecar for the rest; two consumers, cuts against uniformity). Full analysis: `chamber-library/docs/canonical-format-decision-FOR-JURIST-2026-07-12.md`.
+**Recommendation & steward decision:** **Option A + Adopt Docling** — selected by the steward on the evidence (2026-07-12, via AskUserQuestion). Reasoning: unblocks execution now, covers the v1 telos (verbatim/provenance/structure/anchors), preserves inspectability, keeps TEI migration mechanical.
+**The one flip-condition (for the jurist to test):** if the v1 engine needs *structured critical apparatus (A5) as data* or *DTS-interoperable citation*, revisit toward Option B/TEI. Read: both are later/scholarly needs, not v1 blockers.
+**Files affected (on execution):** conversion runbook + graduation-spec (Docling front-end; sidecar as graduated structural layer) — a v2.x amendment, change-class PROPOSAL; new `.meta.json` sidecar extractor; `ocrmac_pdf`/EPUB cruft-strippers retired; reprocess track B reshaped to Docling→sidecar-extract→verify→graduate.
+**Awaiting:** ~~Jurist ratification~~ **JURIST RATIFIED 2026-07-12** — Option A + D1, on independently-verified reasoning (flip-condition tested directly, NOT triggered for v1: voice-manifest = interpretive-persona not manuscript-variant; the corpus's strongest TEI case, Loeb, needs anchors [B2, which A provides] not apparatus [A5]). **Two schema-gating questions CLOSED with substrate evidence (brief §8):** Q1 — Docling emits flat reference-links not native footnotes (0 `[^N]`, 48 `[*](#…)`, `epub_backend` ignores `epub:type`) → the footnote-recognizer arc is REPOSITIONED as the sidecar's note-extraction layer, not retired; Q2 — the Loeb DSL is typed (`[c dimgray]`/`[c darkmagenta]`/`[sup]`, 2.15M+ spans) and the current extractor *flattens* it → **CARDINAL PRINCIPLE set before schema lock: the extractor PRESERVES source typing into typed sidecar fields, never flattens a source-structured distinction to prose** (makes TEI-migration mechanical-by-construction). **REVIEWED-56 can be placed; sidecar schema unblocked to finalize on these two principles.** (Parallel governance still owed: REVIEWED-55 placement.)