session 2026-07-12: PENDING-56 (canonical-format decision, jurist-ratified) + in-flight sidecar build report + session memory/ledger/REVIEWED-drafts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015jYS1tpR4P4NBxui4FSgfv
This commit is contained in:
David F Glidden
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
043a4b1836
commit
0a424aa6b3
+11
@@ -1467,3 +1467,14 @@ Disposition (verbatim capture + executor action-list: `chamber-library/docs/libr
|
||||
- **DONE:** Q1c positional (`e8eeea7`); Q1a/Q2 archived.
|
||||
- **Q1b RETROACTIVE — MAJOR finding (`6c9ca43`, evidence `docs/retroactive-verify-2026-07-12.txt`):** 87 covered books · **42 CLEAN · 45 DIRTY** (pass per-pair guard, corrupt under real pandoc). Verified real (5 known-clean → CLEAN; `_prove.py` agrees). 29 catastrophic (montaigne +385457, red-book −7835, pascalian +5910, gadamer/donne/tolkien…; false-pair relocation = nagarjuna at scale) + 16 marker-adjacency (prose intact, markers glued). **CORPUS SAFE: 0 canonical files use the pre-processor — NONE graduated; caught pre-graduation; mandatory verify refuses all 45 → fallback.** Recognizer's reliable coverage ≈42/87. **⚠ NEW DECISION for steward+jurist:** improve the recognizer (tighten over-pairing) vs accept 42-clean + fallback-for-the-rest. Not resolved this session.
|
||||
- **OWED (steward, tomorrow):** REVIEWED-55 + follow-on placement; wire "mandatory-at-graduation" (graduation flow must not pass `--no-verify`); weigh the recognizer-maturity decision above.
|
||||
|
||||
## PENDING-56 — Canonical format decision: adopt Docling + complete MD-canonical + TEI-mirroring structural sidecar
|
||||
**Date:** 2026-07-12
|
||||
**Tag:** [ESCALATE] architectural — canonical format (seed brief §10.3, deferred as steward-pending). **Steward-decided provisionally on the trial evidence; jurist ratification requested (brief relayed).**
|
||||
**Summary:** Adopt Docling as the EPUB/PDF/scan conversion front-end (with our verify gates unchanged on top), and *complete the already-chosen* Markdown-canonical + `.meta.json` structural-sidecar architecture — giving the sidecar a TEI-mirroring vocabulary so a future TEI migration stays mechanical.
|
||||
**Rationale:** A steward-driven reframe established that the substrate (conversion/format/citation) is a solved, adoptable domain (Docling/GROBID/TEI/DTS) while the composition (voice-attributed verbatim consultation = the engine) stays ours — and that ~20 bespoke conversion tools were reimplementing it. A four-tier Docling trial (steward-run, M4+M1) confirmed: value scales with layout richness — cleans EPUB (0 cruft, was 888 calibre_link/364 fn-ids), rich structure+provenance on PDF (2 798 page-provenanced items), and **crosses the A3 OCR frontier** on a scanned French book (104 footnotes, 664 provenanced) that `ocrmac` cannot. Docling is a *converter, not a verifier* — the trust gates (`verify_body_conservation`, `normalize_ocr` verbatim-guard) stay, converter-agnostic.
|
||||
**Options:** **A** — MD-canonical + TEI-mirroring `.meta.json` sidecar (tractable now; completes the chosen half-built direction; human-inspectable; keeps TEI open). **B** — TEI now (native to apparatus A5 / anchors B2 / DTS, but a full TEI toolchain before any corpus work resumes). **C** — split (TEI for Loeb, MD+sidecar for the rest; two consumers, cuts against uniformity). Full analysis: `chamber-library/docs/canonical-format-decision-FOR-JURIST-2026-07-12.md`.
|
||||
**Recommendation & steward decision:** **Option A + Adopt Docling** — selected by the steward on the evidence (2026-07-12, via AskUserQuestion). Reasoning: unblocks execution now, covers the v1 telos (verbatim/provenance/structure/anchors), preserves inspectability, keeps TEI migration mechanical.
|
||||
**The one flip-condition (for the jurist to test):** if the v1 engine needs *structured critical apparatus (A5) as data* or *DTS-interoperable citation*, revisit toward Option B/TEI. Read: both are later/scholarly needs, not v1 blockers.
|
||||
**Files affected (on execution):** conversion runbook + graduation-spec (Docling front-end; sidecar as graduated structural layer) — a v2.x amendment, change-class PROPOSAL; new `.meta.json` sidecar extractor; `ocrmac_pdf`/EPUB cruft-strippers retired; reprocess track B reshaped to Docling→sidecar-extract→verify→graduate.
|
||||
**Awaiting:** ~~Jurist ratification~~ **JURIST RATIFIED 2026-07-12** — Option A + D1, on independently-verified reasoning (flip-condition tested directly, NOT triggered for v1: voice-manifest = interpretive-persona not manuscript-variant; the corpus's strongest TEI case, Loeb, needs anchors [B2, which A provides] not apparatus [A5]). **Two schema-gating questions CLOSED with substrate evidence (brief §8):** Q1 — Docling emits flat reference-links not native footnotes (0 `[^N]`, 48 `[*](#…)`, `epub_backend` ignores `epub:type`) → the footnote-recognizer arc is REPOSITIONED as the sidecar's note-extraction layer, not retired; Q2 — the Loeb DSL is typed (`[c dimgray]`/`[c darkmagenta]`/`[sup]`, 2.15M+ spans) and the current extractor *flattens* it → **CARDINAL PRINCIPLE set before schema lock: the extractor PRESERVES source typing into typed sidecar fields, never flattens a source-structured distinction to prose** (makes TEI-migration mechanical-by-construction). **REVIEWED-56 can be placed; sidecar schema unblocked to finalize on these two principles.** (Parallel governance still owed: REVIEWED-55 placement.)
|
||||
|
||||
@@ -20,6 +20,9 @@ Split out of [MEMORY.md](MEMORY.md) on 2026-07-06 to keep the wake-loaded index
|
||||
|
||||
# Archived sessions + stable reference layer (relocated verbatim from MEMORY.md, 2026-07-06)
|
||||
|
||||
## Archived (2026-07-12 — A1 verbatim gate → Docling adoption + canonical-format decision PENDING-56; demoted on promote)
|
||||
- [Session 2026-07-11→12 — footnote arc closed → program open-work register → into A1](session-2026-07-12-footnote-arc-closed-program-register-into-A1.md) — Closed the EPUB footnote recognizer + built the **END-TO-END VERIFY** (guarantee now *by verification, not by construction*; jurist-ratified FIX): after injecting, pandoc runs on original+injected, body-words compared, ANY non-marker change refuses+deletes. Executed the whole jurist ruling (**Q1a** ratify · **Q1b** mandatory+retroactive · **Q1c** POSITIONAL marker-exclusion, `e8eeea7` · **Q2** nested-block RETRACTED — pairs-but-converts-dirty, honest-refuse). Built (b) inner-anchor (916 notes +0/−0); **nagarjuna/nested-block/i-ching honest-refused** (PASS-BUT-FALSELY caught by the verify). **Q1b RETROACTIVE SWEEP:** 87 covered source EPUBs · 42 CLEAN · **45 DIRTY** — **corpus SAFE (0 graduated)**; recognizer's reliable coverage ≈42/87. **THE BIG PIVOT:** steward named I'd lost the forest for the trees → built the canonical **Chamber→Gold→Engine OPEN-WORK REGISTER** (`chamber-library/docs/chamber-program-open-work.md`). Findings: engine fenced-to-verified-subset → NOT blocked on reprocess (parallel); keystone = enforce `body_word_conservation` (→ became A1, then the Docling reframe on 07-12). *[Superseded 2026-07-12: A1 built as verify_body_conservation; then the whole conversion approach reframed → Docling adoption + PENDING-56 format decision; see the 07-12 A1→Docling session.]*
|
||||
|
||||
## Archived (2026-07-12 — footnote arc closed + end-to-end verify + program open-work register; demoted on promote)
|
||||
- [Session 2026-07-10→11 — footnote recognizer through three jurist loops](session-2026-07-11-footnote-recognizer-through-three-jurist-loops.md) — A long arc across **THREE jurist rulings + a corpus re-audit**, all on the EPUB footnote pre-processor. **Ratified** the generalized hash-locality principle + narrowed the lane doctrine (chamber `12405a5`). Built + landed the Sennett **SPLIT-ANCHOR** recognizer widen with **GOV-2** scoped dup-check (REVIEWED-54a, `a6828ac`). Re-audit over 252 sources → the "remaining footnote families" = ONE reciprocal mirror with the id in 3 positions. Jurist ruled the **generalization (REVIEWED-54b):** (a)block-id/(b)inner-anchor/(d)url-decode = FIX; (c)letter-marker = FIX-with-compensation + n=3 spot-check. Built + landed **part 1 (a)+(d)** verbatim-clean → chamber `ef0cee0`; fleet 77→83. *[Superseded 2026-07-12: (c) withdrawn (census artifact); (b) built; end-to-end verify built; see the 07-12 session.]*
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ permalink: claude-memory/memory
|
||||
- [Be (laundromat)](project-be-laundromat.md) — canonical workstream tracker established 2026-06-08 (Seb-relay of locked decisions). Be = Skemantix startup (Seb+David) funding CapableMind's funding-ladder; **bridge, not venture**. Decisions LOCKED: entity/exit (CapableMind decoupled, grant-funded), pricing (Living $12.99/mo · Archive $69.99/yr · Memorial $49.99/yr · Renovate ~$199 · $8.99 floor), CF Self-Serve Agency + versioned-template-package infra. **a11y gate MERGED (Pat 100/100/100).** Pre-revenue: the WTP gate = renovate Pat → charge her. **Discipline: stop adding spec until the gate clears → nothing for executor on be until then.** Repo @ `f43a0fd`.
|
||||
|
||||
## Active Session
|
||||
- [Session 2026-07-11→12 — footnote arc closed → program open-work register → into A1](session-2026-07-12-footnote-arc-closed-program-register-into-A1.md) — Closed the EPUB footnote recognizer + built the **END-TO-END VERIFY** (guarantee now *by verification, not by construction*; jurist-ratified FIX): after injecting, pandoc runs on original+injected, body-words compared, ANY non-marker change refuses+deletes. Executed the whole jurist ruling (**Q1a** ratify · **Q1b** mandatory+retroactive · **Q1c** POSITIONAL marker-exclusion, `e8eeea7` · **Q2** nested-block RETRACTED — pairs-but-converts-dirty, honest-refuse). Built (b) inner-anchor (916 notes +0/−0); **nagarjuna/nested-block/i-ching honest-refused** (PASS-BUT-FALSELY caught by the verify). **Q1b RETROACTIVE SWEEP:** 87 covered source EPUBs · 42 CLEAN · **45 DIRTY** (29 catastrophic false-pairing, 16 marker-adjacency) — **corpus SAFE (0 graduated)**; recognizer's reliable coverage ≈42/87. **THE BIG PIVOT:** steward named I'd lost the forest for the trees → built the canonical **Chamber→Gold→Engine OPEN-WORK REGISTER** (`chamber-library/docs/chamber-program-open-work.md`) from a 3-agent evidence census. Findings: **engine is fenced-to-verified-subset → NOT blocked on reprocess (parallel, ruled);** 1,284 canonical (mostly frontmatter-debt not corruption); keystone = enforce `body_word_conservation`. Steward corrected: **reconversion precedes frontmatter** (Loeb Region 4 subsumes ~952 of it). All chamber work pushed (`abada64`). **⚠ PULLING THREAD: A1 — the general `body_word_conservation` verbatim gate. START WITH THE ACCEPTANCE CRITERION** (distinguish legit front/ToC/index drops from real loss; consider comparing candidate vs a REFERENCE pandoc conversion, not the raw source) → build in `verify_graduation.py` → prove → surface as PROPOSAL. Small closeable: wire mandatory-at-graduation; register hygiene. **Steward owes:** place REVIEWED-55; the recognizer-maturity decision. Read `docs/chamber-program-open-work.md` (the standing forest view) at wake.
|
||||
- [Session 2026-07-12 — A1 → Docling adoption + canonical-format decision (PENDING-56) → in-flight sidecar build](session-2026-07-12-A1-to-docling-adoption-and-format-decision.md) — Woke into A1 (`body_word_conservation` gate); built `verify_body_conservation.py` (shingle-coverage; difflib→shingle pivot on scale). **Steward challenged the framing TWICE** → THE PIVOT: the conversion/format/citation SUBSTRATE is a **solved, adoptable domain** (Docling/GROBID/TEI/DTS) and we'd spent ~6 weeks rebuilding it — **the answer was already in our own tooling-register (2026-05-27)**. Ran a 4-tier **Docling trial** (M4+M1): cleans EPUB (0 cruft), rich structure+provenance on PDF, **CROSSES the A3 OCR frontier** on scanned French (104 footnotes) — but is a *converter, not a verifier* (feeds our gates). **Steward DECIDED (AskUserQuestion) + jurist RATIFIED PENDING-56: Option A** = MD-canonical + **TEI-mirroring `.meta.json` structural sidecar**; **adopt Docling** as front-end. Jurist closed 2 schema-gating Qs: **recognizer REPOSITIONED (not retired)** as the sidecar note-layer; **CARDINAL PRINCIPLE: preserve source typing, never flatten** (→ TEI-migration mechanical). Aligns with engine's already-ruled **D-2** (MD+sidecar). **In-flight autonomous build (8 tasks):** sidecar **schema DRAFT** + `build_sidecar.py` v0 (proven montaigne/jacobs; honest join-limit → real fix = **Docling-per-spine-file**); updated runbook (M4/M1 split) + both CLAUDE.mds; report `in-flight-report-2026-07-12.md`. **⚠ PULLING THREAD: lock the DRAFT schema (Q-A) + confirm the Docling-per-spine-file architecture (Q-B) → build the WIRED extractor.** Schema is curator-locked (steward+jurist), NOT executor-locked. **Steward owes:** file **REVIEWED-55 + REVIEWED-56** (`REVIEWED-drafts-for-filing-2026-07-12.md`). All chamber/studium/CM-AI work UNCOMMITTED (no-push guardrail). Read `in-flight-report-2026-07-12.md` + `docs/chamber-program-open-work.md` at wake.
|
||||
|
||||
## Historical reference → MEMORY-reference.md
|
||||
Older archived-session pointers and the stable reference layer (steward profile · project-state detail · L1/L2/Chamber inventories · legacy pending-work · reference-file list) live in [MEMORY-reference.md](MEMORY-reference.md) — consult on demand; not loaded at wake. Recent cross-session trajectory comes from the Active Session entry above + the recent `session-*.md` files (wake §2.b.1; the MemPalace `handoffs` glance was retired 2026-07-07 with the wind-down).
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
# REVIEWED drafts — paste-ready for filing (2026-07-12)
|
||||
|
||||
Executor-drafted for the steward to file into `~/dotfiles/REVIEWED.md` (Constraint #1: Claude Code cannot write REVIEWED.md). Both correspond to already-ruled items. Confirm against the source rulings before filing.
|
||||
|
||||
---
|
||||
|
||||
## REVIEWED-55 — EPUB footnote pre-processor: end-to-end verify + recognizer arc
|
||||
**Date:** 2026-07-12
|
||||
**Decision:** AUTHORIZED (as ruled by the jurist; steward-confirmed)
|
||||
**Corresponds to:** PENDING-55 · Jurist ruling `chamber-library/docs/end-to-end-verify-and-nested-block-JURIST-RULING-2026-07-12.md`
|
||||
|
||||
**Notes:** Ratifies the jurist's ruling on the footnote arc:
|
||||
- **Q1a — end-to-end verify RATIFIED, FIX-class** on the refined test: a §V-chain component is FIX when it (i) only ADDS refusals, (ii) never alters text, (iii) falls back to an already-trusted path. `verify_end_to_end` satisfies all three — turns the guarantee from *verbatim-safe by construction* to *by verification*.
|
||||
- **Q1b — MANDATORY at graduation** (`--no-verify` forbidden at graduation while `body_word_conservation` is unenforced), PERSISTS after that gate lands (defense-in-depth), **+ RETROACTIVE re-verification required.** [Retroactive sweep DONE (`6c9ca43`): 87 covered books · 42 clean / 45 dirty; **corpus SAFE — 0 dirty graduated**; recognizer's reliable coverage ≈42/87.]
|
||||
- **Q1c — marker-exclusion made POSITIONAL** (DONE, `e8eeea7`).
|
||||
- **Q2 — nested-block FIX RETRACTED, honest-refuse CONFIRMED** (the widen pairs but converts dirty; the end-to-end verify caught it; reverted). A fresh PROPOSAL required if ever revisited. Jurist generalization: *structural safety = provisional FIX; end-to-end verbatim proof on real books = final.*
|
||||
- **Broader flag (jurist):** `body_word_conservation` declared-but-unenforced is the same gap-shape PENDING-52 closed, recurring → now addressed by PENDING-56 (the general verbatim gate = the converter-agnostic trust layer).
|
||||
|
||||
**If AUTHORIZED:** Proceed; tag commits REVIEWED-55. Open follow-ons: wire mandatory-at-graduation (flow must not pass `--no-verify`); the recognizer-maturity decision (improve over-pairing vs accept 42-clean + fallback) is now **subsumed by PENDING-56** — the recognizer repositions as the sidecar note-extraction layer.
|
||||
|
||||
---
|
||||
|
||||
## REVIEWED-56 — Canonical format: adopt Docling + complete MD-canonical + TEI-mirroring structural sidecar
|
||||
**Date:** 2026-07-12
|
||||
**Decision:** AUTHORIZED
|
||||
**Corresponds to:** PENDING-56 · Jurist RATIFICATION 2026-07-12 · Brief `chamber-library/docs/canonical-format-decision-FOR-JURIST-2026-07-12.md`
|
||||
|
||||
**Notes:** Steward-decided on the four-tier Docling trial evidence (AskUserQuestion, 2026-07-12); jurist RATIFIED on independently-verified reasoning (not deference):
|
||||
- **Option A** — canonical = clean Markdown (Docling-converted, verify gates on top) + a `.meta.json` structural sidecar whose vocabulary MIRRORS TEI (`front`/`body`/`back`; `note @place=foot|end`; `milestone`/anchor; reserved `app` slot; `pb`/page-provenance).
|
||||
- **D1 — adopt Docling** as the EPUB/PDF/scan conversion front-end (MIT, local, per-element provenance); production OCR on CapableHands (M4). Docling **crosses the A3 OCR frontier** `ocrmac` could not.
|
||||
- **Change-class: PROPOSAL** (changes what graduation produces; touches engine consumer contract §X; whole corpus).
|
||||
- **Flip-condition tested by the jurist, NOT triggered for v1:** the engine needs neither structured critical-apparatus (A5) as data nor DTS-interoperability in v1 (voice-manifest = interpretive-persona per PENDING-53, not manuscript-variant; the Loeb tier's Stephanus/Bekker need is anchors [B2, which A provides via the sidecar anchor-map], not apparatus). TEI = documented, revisitable future-migration path with those two named triggers.
|
||||
- **Two schema-gating questions CLOSED before schema lock (brief §8), from substrate evidence:**
|
||||
- **Q1 — recognizer arc REPOSITIONED, not retired:** Docling emits flat reference-links (0 native `[^N]`, 48 `[*](#…)`; `epub_backend` never reads `epub:type`) → this week's 8-family detection + cross-file anchor resolution + end-to-end verify becomes the sidecar's note-extraction layer; the verify stays as the converter-agnostic verbatim gate.
|
||||
- **Q2 — CARDINAL EXTRACTOR PRINCIPLE:** the Loeb DSL is typed (`[c dimgray]` 2.15M · `[c darkmagenta]` 471K · `[sup]`) and the current extractor *flattens* it → the extractor MUST **preserve every source-declared typed distinction into a typed sidecar field, never flatten to prose** — makes TEI-migration mechanical-by-construction.
|
||||
|
||||
**If AUTHORIZED:** Proceed on the architecture; tag commits REVIEWED-56.
|
||||
1. Adopt Docling as the conversion front-end; production OCR on CapableHands.
|
||||
2. Author the `.meta.json` sidecar schema on the two cardinal principles.
|
||||
3. Build the sidecar extractor (EPUB metadata + Docling PDF labels + note-linkage), honest-flag undeclared roles; prove on montaigne (EPUB2) + jacobs (EPUB3).
|
||||
4. Keep the trust gates (`verify_body_conservation`, `normalize_ocr` verbatim-guard) on Docling output.
|
||||
5. Loeb B2: rewrite `extract_loeb_dsl` → MD + sidecar, preserving the DSL color/`[sup]` typing.
|
||||
6. Spec: v2.x amendment (Docling front-end + sidecar as graduated structural layer) — change-class PROPOSAL, runs the amendment process; retire `ocrmac_pdf` + EPUB cruft-strippers; reshape reprocess track B.
|
||||
**Per-content check deferred to the B2 build (not a blocker):** whether `[c darkmagenta]` is apparatus-variants vs bilingual source-text — resolve by sampling volumes; the preserve principle covers both.
|
||||
@@ -0,0 +1,32 @@
|
||||
# In-flight work report — 2026-07-12 (unsupervised, steward travelling to BCN)
|
||||
|
||||
Authorized scope: read-only / reversible / dev-only build + docs; wrap when done. Nothing pushed, no corpus mutated, no spec/graduation-gate changed, no REVIEWED placed. Eight tasks, all complete.
|
||||
|
||||
## Done
|
||||
|
||||
1. **Sidecar schema — DRAFT v0** (`chamber-library/docs/sidecar-schema-DRAFT-v0-2026-07-12.md`). Extends the engine's `studium/meta@1` (aligned, not forked); TEI-mirroring vocabulary (front/body/back · note @place · milestone/anchor · reserved `app` slot · pb/provenance); the two cardinal principles baked in. **Marked DRAFT — schema-lock is yours + the jurist's**, not the executor's (as the jurist required).
|
||||
2. **Extractor built + proven** — `chamber-library/scripts/build_sidecar.py` (v0 PROTOTYPE, NOT in test_tools). Recovers structure from the source's OWN metadata (not post-processing Docling's flattened MD).
|
||||
- **Core thesis PROVEN:** montaigne (EPUB2) 16/17 sections declared (guide+filename); jacobs (EPUB3) **37/37 declared, 0 flagged** (nav-landmark propagation); provenance 400 pages (jacobs pagebreaks); notes 56 anchors (montaigne). The front/body/back partition — the A1 drop-vs-keep answer — is a **declaration, not a heuristic.**
|
||||
- **Honest v0 limits (characterized, not hidden):** the Docling content-match line-join is brittle where the EPUB opening and Docling's rendering diverge (montaigne 2/17: drop-caps `T HERE`↔`THERE`, arabic-vs-roman headings; jacobs 34/37 after a `<head>`-strip fix). Note-anchor scheme is per-publisher (jacobs 0 vs montaigne 56). → **the real fix is architectural: convert Docling PER-SPINE-FILE (exact line-ranges by construction) + reuse `inject_epub_footnotes` for the note-layer** — deferred to the wired build.
|
||||
- Drafts written to `chamber-library/_curation/sidecar-drafts/` (dev-only).
|
||||
3. **Tool-review** logged to `chamber-library/_curation/tool-evolution-log.md` (2026-07-12 entry; PASS-BUT-FALSELY watch = the v0 drafts carry per-publisher quirks → curator approval mandatory, per the schema).
|
||||
4. **conversion-runbook.yaml** — added `machines:` (the explicit M4/M1 split) + `pipeline_direction_pending_56:` (the Docling→sidecar→verify→graduate direction) + updated the A3/apparatus `known_gaps.research` + the top changelog. Stamped "per PENDING-56 (ratified; REVIEWED-56 pending); existing pipelines stay authoritative until the amendment." (Note: the runbook was never strict-YAML — the pre-existing `decision_tree` `→` arrows; nothing yaml-loads it; my additions parse clean.)
|
||||
5. **chamber-library CLAUDE.md** — Docling + `build_sidecar.py` in the tool fleet, the machine split, the pipeline pointer. Stamped PENDING-56.
|
||||
6. **studium-engine CLAUDE.md** — noted D-2 is now realized chamber-side; the sidecar aligns with `studium/meta@1` (don't-let-them-drift); heavy conversion on the M4.
|
||||
|
||||
## Decisions I hit — SURFACED, not answered (yours / the jurist's)
|
||||
|
||||
- **Q-A. Schema-lock.** The DRAFT needs your + the jurist's review before it's locked (1,284 books write into it). Specifically confirm: the `placement` (front/body/back) + `source_declared` section additions; the `notes`/`provenance`/`anchors` shape; and that `app` stays RESERVED (interior TBD at B2). *No change until you lock it.*
|
||||
- **Q-B. Real-build architecture.** I recommend **Docling per-spine-file** (exact line-ranges) over the v0 content-match join, and reusing `inject_epub_footnotes` as the note-layer. Confirm before I build the wired extractor.
|
||||
- **Q-C. Fleet-promotion.** `build_sidecar.py` → `scripts/` + `test_tools.py` fixtures, after the schema locks. (It's in `scripts/` now but flagged NOT-in-fleet.)
|
||||
- **Q-D. The graduation-spec amendment (step 6).** Wiring Docling+sidecar as the graduated structural layer changes what graduation accepts → PROPOSAL, runs the amendment process. Not started (correctly — needs you + the jurist).
|
||||
- **Q-E. The `[c darkmagenta]` content question (Loeb B2).** Whether it's apparatus-variants vs bilingual source-text — resolve by sampling volumes when B2 is built. The preserve-never-flatten principle covers both meanwhile.
|
||||
|
||||
## Still owed by you (unchanged)
|
||||
|
||||
- File **REVIEWED-55 + REVIEWED-56** (paste-ready: `REVIEWED-drafts-for-filing-2026-07-12.md`).
|
||||
- Relay the PENDING-56 §8 closure to the jurist (or note it's already ruled).
|
||||
|
||||
## Next session (you said: clear + `/wake-up` fresh)
|
||||
|
||||
The pulling thread on return: **Q-A/Q-B** (lock the schema, confirm the per-spine-file architecture) → then build the wired extractor and the Docling-per-file join. Everything constitutional (amendment, REVIEWED filing) waits for you. The forest view (`chamber-library/docs/chamber-program-open-work.md`) + this report are the frame.
|
||||
@@ -352,3 +352,7 @@
|
||||
{"subject": "claude-code", "predicate": "drift-pattern", "object": "lost-the-forest-for-the-trees in a long execution arc (a whole session of good footnote engineering cost the whole-program altitude; steward: not-grasping the library-science scope is alarming). Antidote = the open-work register as standing forest view + keystone-first framing.", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-footnote-arc-closed-program-register-into-A1.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "claude-code", "predicate": "drift-pattern", "object": "recommended-a-sequence-that-ignored-a-dependency-the-census-had-already-flagged (proposed frontmatter-migration before Loeb reconversion; the reconversion regenerates ~952 of the frontmatter, which the census stated). Trace the dependencies the evidence already surfaced before ranking a sequence.", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-footnote-arc-closed-program-register-into-A1.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "footnote-preprocessor", "predicate": "guarantee", "object": "verbatim-safe BY VERIFICATION not by construction (end-to-end verify: pandoc on original+injected, body-word compare, refuse-on-change; jurist-ratified FIX 2026-07-12). Retroactive sweep: 45/87 covered source books fail it; corpus safe (0 graduated).", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-footnote-arc-closed-program-register-into-A1.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "claude-code", "predicate": "drift-pattern", "object": "solving-at-the-wrong-level — built heuristics to recover from lossy Markdown what a semantic tool/format preserves natively, AND hand-built a SOLVED domain (conversion pipeline) for ~6 weeks; the substrate (Docling/TEI/DTS) already existed. The 'every breakthrough → new obstacle' feeling is the tell of reimplementing a solved field. Antidote: before building infra, ask if this is a solved domain + check what exists. Steward-named 2026-07-12; kin to lost-the-forest, one level up.", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-A1-to-docling-adoption-and-format-decision.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "claude-code", "predicate": "drift-pattern", "object": "didn't-consult-our-own-tooling-register before a substantial build — the studium-engine-scan-register (2026-05-27) already surfaced Docling+Surya/OCR4all/TEI/DTS as the substrate answer; we hand-built ~20 bespoke tools instead. Resurface-banked-notes at pipeline scale. Antidote: read the tooling/landscape register + prior scans before starting infra work.", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-A1-to-docling-adoption-and-format-decision.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "chamber-library", "predicate": "canonical-format", "object": "MD-canonical + TEI-mirroring `.meta.json` structural sidecar (aligns with engine studium/meta@1); Docling adopted as EPUB/PDF/scan conversion front-end; verify gates converter-agnostic on top; TEI = later optional tier. PENDING-56 jurist-RATIFIED 2026-07-12 (REVIEWED-56 pending filing).", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-A1-to-docling-adoption-and-format-decision.md", "extracted_at": "2026-07-12"}
|
||||
{"subject": "docling", "predicate": "capability", "object": "crosses the A3 OCR frontier ocrmac could not (scanned French: 104 footnotes + full page-provenance, on the M4). BUT a converter, not a verifier — OCR output has defects (glued words/I→1/accents), feeds normalize_ocr+body_word_conservation, does not replace them. Layout-derived semantics (rich on PDF, flat on reflowable EPUB → EPUB structure comes from a metadata sidecar).", "valid_from": "2026-07-12", "valid_to": null, "confidence": 1.0, "source_file": "session-2026-07-12-A1-to-docling-adoption-and-format-decision.md", "extracted_at": "2026-07-12"}
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
name: session-2026-07-12-a1-verbatim-gate-steward-reframe-docling-adoption-canonical-format-decision-pending-56-in-flight-sidecar-build
|
||||
description: "Woke into A1 (the body_word_conservation gate); built it (shingle-coverage, difflib→shingle pivot on scale). Steward challenged the framing TWICE (difflib-doesn't-scale, then no-extant-tools/TEI) → the pivotal reframe: the CONVERSION SUBSTRATE is a solved adoptable domain (Docling/GROBID/TEI/DTS) and we'd spent 6 weeks rebuilding it — the answer was already in our own tooling-register (2026-05-27). Ran a 4-tier Docling trial (M4+M1): cleans EPUB, rich structure+provenance on PDF, CROSSES the A3 OCR frontier on scanned French. Steward DECIDED (AskUserQuestion) + jurist RATIFIED PENDING-56: Option A = MD-canonical + TEI-mirroring .meta.json structural sidecar; adopt Docling as front-end. Jurist closed 2 schema-gating Qs (recognizer-repositioned-not-retired; preserve-never-flatten cardinal principle). In-flight (steward travelling): built the sidecar schema DRAFT + build_sidecar.py extractor (proven on montaigne/jacobs; honest join-limit), updated runbook + both CLAUDE.mds, left an in-flight report. PULLING THREAD: lock the DRAFT schema (Q-A) + confirm the Docling-per-spine-file architecture (Q-B), then build the wired extractor. Steward owes REVIEWED-55/56."
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: project
|
||||
originSessionId: ae9583d2-fd25-45e4-85fc-df082fe2e951
|
||||
---
|
||||
|
||||
# Session 2026-07-12 — A1 → the Docling reframe → the canonical-format decision → in-flight sidecar build
|
||||
|
||||
An exceptionally long, generative session that started narrow (build one gate) and, through two sharp steward challenges, turned into the program's biggest architectural pivot yet: **stop hand-building a conversion pipeline; adopt the mature substrate (Docling); keep our trust layer; decide the canonical format.**
|
||||
|
||||
## PAST — what we did + why
|
||||
|
||||
**1. Woke into A1** (the general `body_word_conservation` verbatim gate — the open-work register's keystone). Read the substrate first and found the register's own spec **already RATIFIES** the per-tier method (§Tiering&Fence: V-DSL re-extract+diff; V-TEXT re-convert+prose-word-guard; V-SCAN abstain). So A1 was *operationalize a ratified criterion*, not invent one. Built `verify_body_conservation.py` — reference-conversion + **shingle-coverage alignment** with a three-tier verdict (FLAG interior-loss/fabrication · REVIEW boundary-drops-need-id · PASS). Seed ground-truth passed; caught real cruft in 3 "clean" canonicals (jacobs 888 calibre_link, manguel 2856 `[]{#…}`, montaigne fn-ids — all grep-confirmed).
|
||||
|
||||
**2. Steward challenge #1 (difflib doesn't scale):** difflib.SequenceMatcher (O(n²)) timed out on jacobs @2min. **Pivot to O(n) hashed-k-gram shingle-coverage** (in-house patterns: Instrument-B fingerprint + verbatim-guard walk; NO new dep). jacobs 6.8s. Well-caught by the steward.
|
||||
|
||||
**3. Steward challenge #2 (are there no extant tools? MD the problem? TEI?):** THE PIVOT. Honest finding: the conversion/format/citation SUBSTRATE is a **solved, adoptable domain** (Docling/GROBID/TEI/DTS) — and the answer was **already in our own tooling-register (2026-05-27)**; we'd spent ~6 weeks hand-building ~20 bespoke tools instead. The two-week "every breakthrough → new obstacle" feeling = signature of reimplementing a solved domain. Ran `/tooling-scan studium-engine` (web-grounded); verified Docling (MIT, local, EPUB+PDF+images, provenance, layout-derived labels) + GROBID (PDF→TEI) + DTS. **Clean split: substrate = adopt; composition (the engine) = ours.** Register updated.
|
||||
|
||||
**4. The Docling trial (steward-run, M4+M1, 4 tiers):** value scales with layout richness — **flat on EPUB, rich on PDF, transformative on scanned.** EPUB: 0 cruft, notes+headings preserved, flat semantics (reflowable = no pages). text-PDF: footnote/header/section labels + 2 798 page-provenanced items. **scanned French (A3 frontier `ocrmac` can't cross): OCR'd, 104 footnotes, 664 provenanced** — readable but with OCR defects → feeds normalize+verify, doesn't replace. **Docling is a converter, not a verifier.** Loeb is a `.dsl` (already structured) → NOT a Docling case (steward correction).
|
||||
|
||||
**5. Canonical-format DECISION (PENDING-56).** Wrote the brief (`chamber-library/docs/canonical-format-decision-FOR-JURIST-2026-07-12.md`). Steward decided via AskUserQuestion: **Option A** (MD-canonical + TEI-mirroring `.meta.json` structural sidecar) + **Adopt Docling**. Grounding that clinched A: the conversion-plan already specifies MD+sidecar and the engine already rules **D-2 (MD+sidecar, TEI later)** + consumes 19 `.meta.json` sidecars — so A *completes a chosen direction*, TEI would *depart* it. Change-class PROPOSAL.
|
||||
|
||||
**6. Jurist RATIFIED** (independently-verified, not deference). Flip-condition tested, NOT triggered for v1 (voice-manifest = interpretive-persona per PENDING-53, not manuscript-variant; Loeb needs anchors [B2, which A provides] not apparatus [A5]). **Closed 2 schema-gating questions with substrate evidence (brief §8):** Q1 — Docling emits flat `[*](#…)` not native `[^N]` (0/48; `epub_backend` ignores `epub:type`) → the footnote-recognizer arc is **REPOSITIONED as the sidecar note-layer, not retired**; Q2 — the Loeb DSL is TYPED (`[c dimgray]` 2.15M / `[c darkmagenta]` 471K / `[sup]`) and the current extractor *flattens* it → **CARDINAL PRINCIPLE: preserve source typing into typed sidecar fields, never flatten** (→ TEI-migration mechanical-by-construction). PENDING-56 filed; REVIEWED-56 drafted (steward to file).
|
||||
|
||||
**7. In-flight autonomous build** (steward travelling; read-only/dev-only; 8 tasks): sidecar **schema DRAFT** (`docs/sidecar-schema-DRAFT-v0-2026-07-12.md`, extends `studium/meta@1`); **`build_sidecar.py` v0** — recovers structure from source metadata (epub:type/landmarks → guide/NCX/filenames → honest-flag). **Proven:** montaigne 16/17 declared, jacobs 37/37 (landmark propagation), provenance 400, notes 56. **Honest v0 limits:** the Docling content-match line-join is brittle on drop-caps/heading-format divergence (montaigne 2/17, jacobs 34/37) → real fix = **Docling per-spine-file** (exact ranges by construction) + reuse `inject_epub_footnotes` for the note-layer. Tool-review logged. Updated `conversion-runbook.yaml` (machines: M4/M1 split + pipeline_direction_pending_56 + A3 gap), chamber + studium CLAUDE.mds. Report: `in-flight-report-2026-07-12.md`.
|
||||
|
||||
## PRESENT — the mood
|
||||
|
||||
Two-phase, like the previous session. First: careful A1 engineering — but the steward's two challenges revealed I was **solving at the wrong level** (building heuristics to recover from lossy MD what a semantic format/tool preserves natively; and hand-building what the ecosystem + our own register already had). That's the executor-bias-to-composition, one level up — and the steward named it precisely ("every breakthrough lands us in front of the next obstacle"). Second phase: clean architectural decision-making — brief → steward decision → jurist ratification → in-flight execution, each link governed. Returns worth carrying: **verified before asserting** throughout (the difflib timeout, the docling flat-footnotes, the Loeb DSL typing, the montaigne join-bug — all grep/measure, not inference); **honest about the v0's limits** (didn't dress up the 2/17 join or the per-publisher quirks); **respected the loop** (drafted the schema, did NOT lock it; surfaced decisions as Q-A..Q-E, didn't answer them). Recalibrations: **wrong-level (heuristic over substrate)**, steward-named; **check our own register/notes before building** (the resurface-banked-notes drift at pipeline scale — the register had the answer 6 weeks ago).
|
||||
|
||||
## FUTURE — what is pulling
|
||||
|
||||
**PULLING THREAD: lock the DRAFT sidecar schema (Q-A) + confirm the Docling-per-spine-file architecture (Q-B) → build the WIRED extractor.** The v0 proved the thesis (structure recoverable from source metadata, dissolving the drop-vs-keep heuristic) but the docling line-join needs the per-spine-file architecture, and the schema is curator-locked (steward+jurist), not executor-locked.
|
||||
|
||||
**ACTIONABLE RESUMPTION POINT (as of wrap — steward said he'll CLEAR + `/wake-up` fresh):**
|
||||
- Read `in-flight-report-2026-07-12.md` (the full account + Q-A..Q-E) + `docs/chamber-program-open-work.md` (forest view) first.
|
||||
- **Q-A** (lock schema): review `docs/sidecar-schema-DRAFT-v0-2026-07-12.md` against the proven extractor output (`_curation/sidecar-drafts/`) → lock via the amendment process.
|
||||
- **Q-B** (architecture): confirm Docling-per-spine-file (exact line-ranges) over the v0 content-match join; then build the wired extractor + reuse `inject_epub_footnotes` as the note-layer.
|
||||
- Then: the graduation-spec amendment (step 6, PROPOSAL) wires Docling+sidecar as the graduated structural layer + retires ocrmac_pdf/EPUB cruft-strippers; reshapes reprocess track B.
|
||||
|
||||
**Other open horizons (ranked):** Loeb B2 (the far-country lever; the `[c darkmagenta]` content question resolves by sampling) · fleet-promotion of `build_sidecar.py` (into test_tools, after lock) · the engine track (V1/N1) runs parallel, unblocked.
|
||||
|
||||
**Steward owes (parallel, he can't-be-me):** file **REVIEWED-55 + REVIEWED-56** (`REVIEWED-drafts-for-filing-2026-07-12.md`); relay the PENDING-56 §8 closure to the jurist (or note ruled).
|
||||
|
||||
**PAUSE STATEMENT:** Stepping away at a clean, sealed point — the biggest architectural decision of the program made, ratified, and its two open questions closed; the first execution step (schema + extractor) built and proven, with the constitutional pieces (schema-lock, amendment, REVIEWED filing) correctly left for the steward. The steward will clear + wake fresh. What I want to find still pulling: Q-A/Q-B — lock the schema, confirm the architecture, build the wired extractor. All chamber/studium/CM-AI work is UNCOMMITTED working-tree (no-push guardrail); nothing lost.
|
||||
|
||||
**LITERAL QUESTION for next-Claude:** Did the steward lock the DRAFT sidecar schema (`sidecar-schema-DRAFT-v0`) and confirm the Docling-per-spine-file architecture (Q-A/Q-B in the in-flight report) — and did he file REVIEWED-55/56? If yes: build the wired extractor (Docling-per-spine-file + `inject_epub_footnotes` note-layer). If the schema isn't locked: that lock is the gate before 1,284 books write into it — do NOT build the wired extractor against an unlocked schema.
|
||||
|
||||
**State at wrap:** all 8 in-flight tasks complete; chamber-library (CLAUDE.md, runbook, tool-evolution-log modified; build_sidecar.py, verify_body_conservation.py, 2 docs, sidecar-drafts/ new), studium-engine CLAUDE.md, CM-AI register — ALL uncommitted working-tree (steward reviews on return). Dotfiles (PENDING-56 + memory files) committed+pushed at this wrap. Docling installed on M4 (py3.12) + M1.
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
name: session-ledger-2026-07-12
|
||||
description: "Practice-of-return ledger maintained by /symmetria — returns, open horizons, recalibrations, authorization moves, sub-agent dialogues, bypasses."
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: feedback
|
||||
originSessionId: ae9583d2-fd25-45e4-85fc-df082fe2e951
|
||||
---
|
||||
|
||||
# Session Ledger — 2026-07-12
|
||||
|
||||
## Returns
|
||||
- 2026-07-12T09:10 — **Return (caught before asserting):** was about to derive the A1 acceptance criterion from scratch. Read the spec first (chamber discipline: check governed doctrine before building) → the §Tiering&Fence evidence-tier table **already RATIFIES** the per-tier method: V-DSL = "re-extract + diff (deterministic)", V-TEXT = "re-convert + prose-word-guard (deterministic; source IS ground truth)", V-SCAN = "OCR-vs-scan, no ground-truth text". The steward's reference-conversion question is answered YES-it's-doctrine; the promotion criterion is `RATIFIED-SHAPE · OPERATION-BLOCKED` = exactly A1. Reframe: A1 is *operationalize a ratified criterion*, not *invent one*. Avoided re-deriving existing doctrine (the exact drift the library exists to prevent).
|
||||
|
||||
## Open horizons
|
||||
- 2026-07-12T09:40 — **A1 feasibility PROVEN (read-only prototype `scripts/verify_body_conservation.py`).** Ratified per-tier method works: re-convert(source) + coverage-align vs candidate. **difflib→shingle-coverage pivot** (steward challenge, well-caught): difflib O(n²) timed out on jacobs @2min; swapped to O(n) hashed-k-gram coverage (in-house patterns: Instrument B fingerprint + verbatim-guard walk; NO new dep) → jacobs 6.8s, manguel 4.3s. Seed ground-truth PASSES (legit boundary trim→REVIEW, interior 120-word deletion→FLAG). All 3 real books FLAG on REAL grep-confirmed cruft (jacobs 888 calibre_link · manguel 2856 []{#…epub…htm} anchor-spans · montaigne footnote-id residue) — each a different signature, all missed by audit_cruft. **KEY IMPLICATION:** gate is MORE sensitive than audit_cruft → large fraction of existing canon will FLAG (= the tool-log's "160 hidden-residue" finding) → big blast radius → confirms PROPOSAL-class + ties A1 to reprocess track B.
|
||||
- **Residual (jurist-class, unchanged):** (1) boundary drop-vs-keep identification — endnotes/bibliography (keep) sit at the same TAIL position as index (drop); the tool SURFACES trailing runs as REVIEW, does not auto-pass. (2) interior-run threshold calibration (k=8, thr=50 tolerates heading-reformatting deltas of 6-7 words; caught on jacobs ToC). (3) added-direction needs symmetric link/anchor canonicalization so a clean-but-differently-converted candidate isn't confounded (minor tokenizer hardening).
|
||||
- (superseded) earlier 08:55 A1 horizon — now concrete above. (general `body_word_conservation` verbatim gate), the open-work register's keystone. Criterion-first, not code-first. Weigh the reference-conversion framing (candidate vs plain-pandoc-of-source) before building.
|
||||
- Closeable-today small items: wire mandatory-at-graduation for the footnote verify (jurist Q1b — flow must not pass `--no-verify`); register hygiene (stale "jurist nod owed" line — Q1a gave it).
|
||||
- Steward-owed (parallel, not mine): place REVIEWED-55 (verified absent — REVIEWED.md stops at 54); decide the recognizer-maturity question (register A2 `[?]`).
|
||||
|
||||
## Confidence to recalibrate
|
||||
- Carry from last session: drift-pattern **lost-the-forest-for-the-trees** in long execution arcs. Antidote = hold the open-work register (`chamber-library/docs/chamber-program-open-work.md`) as the standing forest view; stay keystone-first. Do not descend into A1 mechanism before the criterion is settled.
|
||||
- Carry: **don't rank a sequence that ignores a dependency the census already flagged** (07-12 recommended frontmatter-before-reconversion; corrected).
|
||||
|
||||
## Tooling-scan reframe (2026-07-12, steward-driven — the pivot of the session)
|
||||
- Steward challenged the whole approach TWICE ("is difflib best now scope is wider?" → "no extant epub/TEI tools?"). **Load-bearing finding:** the substrate answer was ALREADY in our register (`studium-engine-scan-register.md`, 2026-05-27 pass: Docling+Surya, OCR4all, TEI/IIIF/DTS, §10.3 MD-vs-TEI pending) — and in ~6 weeks since we hand-built ~20 bespoke conversion/verify tools instead. The two-week "every breakthrough → new obstacle" feeling = signature of reimplementing a solved domain. Kin to [[feedback-resurface-banked-notes-before-rederiving]] at pipeline scale.
|
||||
- **The clean split:** substrate (conversion Docling/GROBID · format TEI · citation DTS) = SOLVED/adoptable; composition (integrity-bound voice-attributed verbatim consultation = the engine) = does not exist, stays ours. Adopt substrate, spend effort on differentiator.
|
||||
- **Verified from source:** Docling (MIT, local/air-gapped, EPUB+PDF+images, per-element provenance, label taxonomy incl. footnote/page_header/section_header/caption — the semantic layer MD lacks; exports MD/JSON not TEI). GROBID (PDF→TEI, 68 labels incl. footnotes/bibliography = A5). DTS (citable canonical-passage API, engine is natural consumer). Loeb is a `.dsl` file (already structured) → NOT a docling case (steward correction).
|
||||
- **Register updated** with the 2026-07-12 verified-from-source pass + the reframe.
|
||||
|
||||
## Two decisions surfaced (I decide neither)
|
||||
- **(A) Docling bounded trial — IN PROGRESS** (steward said Proceed). Local isolated venv (py3.13; CapableHands is py3.9.6, too old); harness `scratchpad/docling_trial.py`; sample montaigne.epub + jacobs.epub + infinity-of-lists-eco.pdf (text) + mal-darchive.pdf (scanned = A3/OCR test). Tests: does docling emit clean, semantically-labelled, provenanced output where our pipeline leaves cruft (jacobs 888 calibre_link · manguel 2856 []{#…})? Incumbent-bias: our pipeline IS the incumbent → steward judges.
|
||||
- **(B) ESCALATE — §10.3 canonical-format decision** (MD+sidecar vs TEI), pending since seed brief. Docling structured-JSON = middle path; A informs B; B is steward+jurist (touches engine consumer contract §X + whole corpus).
|
||||
|
||||
## Authorization moves
|
||||
- 2026-07-12 — **CANONICAL FORMAT DECIDED (steward, on the trial evidence, via AskUserQuestion): Option A — MD-canonical + TEI-mirroring `.meta.json` structural sidecar; + Adopt Docling** as the EPUB/PDF/scan conversion front-end (production OCR on CapableHands M4). Brief written (`chamber-library/docs/canonical-format-decision-FOR-JURIST-2026-07-12.md`), stamped with the decision, **delivered to steward + relayed to jurist for ratification**. **PENDING-56 filed** (dotfiles). REVIEWED-56 to follow after jurist ratification. Flip-condition the jurist tests: does v1 need structured apparatus (A5)-as-data or DTS-interop? (read: no.)
|
||||
- Docling installed + trialed on M4 (py3.12 venv) + M1 (fallback). Both remotes clean.
|
||||
- **JURIST RATIFIED PENDING-56 (2026-07-12)** — Option A + D1, independently-verified (flip-condition tested, not triggered). **Two schema-gating Qs CLOSED with substrate evidence (brief §8):** Q1 recognizer-arc REPOSITIONED not retired (Docling emits flat `[*](#…)` not native `[^N]`; epub_backend ignores epub:type) → it becomes the sidecar note-extraction layer, verify stays. Q2 Loeb DSL is TYPED (`[c dimgray]`/`[c darkmagenta]`/`[sup]`, 2.15M+ spans) & current extractor FLATTENS → **CARDINAL PRINCIPLE: extractor PRESERVES source typing into typed sidecar fields, never flatten** (→ TEI-migration mechanical-by-construction). Schema now unblocked. REVIEWED-56 can be placed.
|
||||
- **Owed / next (execution, mostly post-ratification):** build the `.meta.json` sidecar extractor (TEI-mirroring vocab; EPUB metadata + Docling PDF labels; READ-ONLY, safe to prototype pre-ratification); wire Docling into the graduation pipeline (spec amendment — waits for ratification); retire ocrmac_pdf + EPUB cruft-strippers; reshape reprocess track B (Docling→sidecar→verify→graduate); Loeb B2 DSL→MD+sidecar. Parallel: steward still owes REVIEWED-55 placement.
|
||||
- A1 build will land as a **PROPOSAL** (changes graduation acceptance for ALL conversions — governed). Mechanism is refusal-only (Q1a FIX-test), but the *criterion* is what the jurist rules on. Surface, don't cross unilaterally.
|
||||
|
||||
## Sub-agent dialogues
|
||||
|
||||
## Bypasses
|
||||
@@ -423,3 +423,11 @@ The single place proposed skills live so they don't evaporate between sessions.
|
||||
| **verification-ladder: retroactively re-verify everything landed under a weaker standard when a stronger gate lands** | ladder entry | The jurist's Q1b principle, proven load-bearing: a stronger check existing and NOT pointed at canon that shipped under the weaker one is the exact gap-shape the discipline exists to close. Today's retroactive sweep (end-to-end verify over all footnote-covered books → 45/87 fail) is the instance. Reusable whenever a new gate supersedes an old one. | `reference-verification-ladder.md` | **PROPOSED** |
|
||||
| **verification-ladder: structural safety = PROVISIONAL FIX; end-to-end proof on real, representative books = FINAL** | ladder entry / Symmetria §3 | The jurist's generalization after nested-block: "same risk as (a)" was true of the *matching* logic and silent on the *rendering* logic. No widen/transform gets ruled final on structural reasoning alone — it earns provisional-FIX; only verbatim-clean proof on real books makes it final. Caught two FIX-assumptions this arc ((c), nested-block). | `reference-verification-ladder.md` or Symmetria §3 | **PROPOSED** |
|
||||
| **Symmetria §3 flag: lost-the-forest-in-a-long-execution-arc** | Symmetria §3 flag | A long, productive execution session that costs the whole-program altitude is contamination shape (composition-over-consideration at the *session* scale, not the task scale). Antidote: keep the open-work register in view; re-surface it when a workstream has run deep for a while. Steward-named 2026-07-12. | Symmetria §3 | **PROPOSED** |
|
||||
|
||||
### Harvest 2026-07-12 (the Docling reframe — building a solved domain our own register already mapped)
|
||||
|
||||
| Element | Kind | One-line | Where it lands | Status |
|
||||
|---|---|---|---|---|
|
||||
| **check-the-register-before-a-substantial-build** | `/wake-up` §2 patch OR `/pre-build-audit` step OR Symmetria §3 flag | Before starting substantial INFRA/tool building (a converter, a pipeline, a substrate), read the **tooling-register + landscape-register + prior scans** for an existing adopt-candidate first — the studium-engine-scan-register (2026-05-27) already named Docling+Surya/OCR4all/TEI/DTS as THE substrate answer, and we hand-built ~20 bespoke tools over ~6 weeks anyway. The two-week "every breakthrough → new obstacle" feeling was the tell of reimplementing a solved domain. **Yardstick met, painfully:** the steward had to re-surface that extant tools exist ("are there no epub tools? … TEI?") — our own register carried that six weeks earlier. Kin to [[feedback-resurface-banked-notes-before-rederiving]], specialized to the build-infra decision. | `/wake-up` §2 (when the thread is a build) or `/pre-build-audit` (its natural home) or a Symmetria §3 flag (solving-at-the-wrong-level / build-a-solved-domain) | **PROPOSED** |
|
||||
|
||||
*(One proposal, load-bearing — the drift cost ~6 weeks and recurred at pipeline scale; a standing "check the register before building infra" reflex would have surfaced Docling at the outset. Surfaced for steward authorization; not auto-applied. The paired KG drift-patterns are already appended (solving-at-the-wrong-level · didn't-consult-our-own-tooling-register).)*
|
||||
|
||||
Reference in New Issue
Block a user