dotfiles/claude/memory/session-2026-06-26-chamber-ocr-pipeline-levi-first-graduation.md

name, description, metadata

name	description	metadata
session-2026-06-26-chamber-library-spec-ratified-ocr-pipeline-levi-first-spec-conformant-graduation	Carried the Chamber Library spec through the full steward+jurist loop (§§I–VI ratified); built the reusable OCR→canonical pipeline (normalize_ocr.py + insert_chapter_headings.py) with a verbatim word-guard; graduated Levi's *The Drowned and the Saved* as the chamber's FIRST spec-conformant text; committed/pushed all 3 repos; re-pointed the engine's levi-gray-zone from the omnibus to the standalone. PULLING THREAD: apply the proven pipeline to the rest of the OCR sources — Alexander Books 1–4 next.	node_type type originSessionId memory project ff1e6982-7c43-4618-9fbe-c5b83ce08e53

Session 2026-06-26 — the Chamber Library got its constitution AND its first conformant text

A long, deep craft session. It began with /wake-up on the held Chamber Library spec (§§II–III drafted, awaiting jurist) and ended with the spec ratified §§I–VI, a proven OCR→canonical pipeline, and the first spec-conformant graduation (Levi) committed and backed up. The day set a precedent.

PAST — what we did + decided

The spec — ratified §§I–VI (steward + jurist loop, run live in conversation)

• Passed the seed spec to the jurist (via the steward); jurist returned a substantive disposition. Executor read the actual spec against it (verify-not-inherit), concurred broadly, added two refinements.
• Ratified + applied to chamber-library/docs/chamber-library-specification.md:
  • §§II–III — source-agnostic canonical form + work + chapter/section + verbatim quote citation, + the jurist's sectionless reduction (a sectionless work cites by work + quote; missing section field = correct, not a defect).
  • §V — the verbatim guarantee as a three-tier normalization (Remove / catalogued-normalization / never-alter-words). Resolved the spec's internal contradiction: it claimed "byte-faithful" while permitting \→---/paragraph-rejoining. New rule: "verbatim" = faithful to the WORDS under declared normalization, NOT byte-identity. Executor refinement folded: minimize the catalogue — prefer preserving source characters; typographic normalization only on concrete need, never prophylactically (tighten-the-verifier, Charter §VII).
  • §VI architecture — two sidecars: reading-index (derived, hash-bound) + voice manifest (authored, engine-side, no hash); voice: <name> stays in corpus as a routing tag; the one corpus-spec invariant: the grounding gate verifies only against canonical text, never a manifest/index. Grounded in the existing reading-indices/*.yaml practice (the steward's "voicing files" — already exactly the jurist's pattern). The reading-index finding (3 layers, navigation≠persona) was relayed to the jurist and shaped the disposition.
  • §X antechamber clause; §VII gate found already conformant.
• PENDING-42 written (dotfiles); REVIEWED-45 owed from steward (loop ran live; Claude Code cannot write REVIEWED.md — constitutional).
• Captured the sidecar typology + orchestration thinking in studium-engine/docs/the-sidecar-typology-and-orchestration-2026-06-26.md (engine-governance; anchored to charter §VI/§VIII/§X/§XIV; the §XIV inquiry-record = the studium↔L1 seam).

The pipeline — built + proven (the reusable precedent)

• scripts/normalize_ocr.py (new, supersedes clean_ocrmac_pagination.py): olmOCR/ocrmac ## Page N → canonical. Drops page markers; conservative seam-rejoin (join only where the page-top begins lowercase — measured: Levi 149 joins / 47 boundaries); dictionary-validated de-hyphenation with a three-way classifier — merge (inflection-aware: countries→country), doubling (fragment is a prefix of next word, com-/companies → auto-drop the fragment, deterministic), compound (lowercase word-fragment + non-word merge, well-/intentioned → keep hyphen, defer to human); running-head→heading; verbatim word-guard (§V Tier-3 proof); conversion record (§V Tier-2). Dry-run default; raw retained off-canon (§III).
• scripts/insert_chapter_headings.py (new): per-book curatorial structure map (TITLE<TAB>anchor), deterministic, fails loud on ambiguous/absent anchor. The curatorial judgment lives in the map; the tool is mechanical → reproducible.
• Key craft caught a PASS-BUT-FALSELY in my own verbatim guard: it passed "within 26 de-hyphenation merges" while 3 actually altered words. Fix = dictionary-validate, not count. The de-hyphenation iterated 10→3→1 deferred (only the real compound, steward-ruled KEEP).

Levi — the FIRST spec-conformant graduation

• canonical_texts/.../the-drowned-and-the-saved-primo-levi.md: §IV/§VI frontmatter, Coleridge epigraph kept, 10 verified chapter headings (Preface + 8 chapters + Conclusion — each located by reading the body against the ToC, verified at its opening), apparatus trimmed front+back, 58,193 words. verify_conversion PASS (cruft 0 / encoding clean / 10 headings / word-sanity).
• Raw page-marked original → converted_texts/.../ (§III page-provenance) + conversion.yaml + structure.tsv.
• Structure-recovery finding (load-bearing for Alexander): this olmOCR output has NO heading markup, NO body page numbers, and the PDFs have NO outline — so chapter structure is recovered curatorially from the ToC, per-book. Confirmed not a Levi quirk — Alexander's 4 books share it (worse).
• The steward's page-number idea (use printed page numbers to anchor structure before stripping): tested against substrate → doesn't stick for current files (olmOCR's clean-prose prompt already discarded all page furniture: 0 page numbers in body). But it's right one layer down → banked as a driver enhancement (preserve page numbers at OCR time) in the runbook known_gaps.next.

Committed + pushed (everything saved/backed up)

• chamber-library 7b46937 (→ skemantix) — spec + pipeline + Levi graduation + governance.
• dotfiles adb3d88 (→ github) — PENDING-42 + ledger.
• studium-engine 6abb593 (thinking doc) + 38de1a9 (corpus WIP + the Levi re-point) (→ github).
• Engine re-point (item 1, DONE): levi-gray-zone manifest + sidecar re-pointed omnibus→standalone (sha 1c66c5d…, 894 lines; Gray Zone now the clean ## chapter L116–257, was hand-walked L28829–28989; source_sha256 re-binds). Station I scope kept Gray-Zone-only (flagged: whole work now trivially available).

PRESENT — mood / returns

• Deep craft session, bidirectional honesty. The steward's framing-challenges worked both ways — his page-number idea I tested (didn't reflexively accept or dismiss) and reported honestly that it doesn't stick + why + where it does. His two-tier framing last session he corrected mine; this session I corrected a wrong premise of my own (fix-2 "running-heads" overturned by measurement, brought home not built wrong).
• Returns (the practice's record): (1) doubling false-defers → iterated the tool rather than accept the noisy 10-deferred result; (2) page-number idea → measured against substrate before verdict; (3) heading-recovery premise → overturned by measurement, surfaced as a design fork not papered; (4) avoided reinventing governed tooling (checked clean_ocrmac_pagination + graduate_to_canonical before building/hand-rolling); (5) the PASS-BUT-FALSELY in my own guard — caught by reading the flagged merges, not trusting the PASS.
• Two permission denials, both correct guardrails, surfaced honestly: the CapableHands driver write + my self-granting a permission rule. Did NOT work around them.
• tool-review-after-each-use earned its keep again — the de-hyphenation iteration is logged in _curation/tool-evolution-log.md (the PASS-BUT-FALSELY is the priority signal).

FUTURE — what is pulling

PULLING THREAD (singular): Apply the proven OCR→canonical pipeline to the rest of the Making/chamber OCR sources — Alexander Books 1–4 next — the graduations that follow the precedent Levi set.

ACTIONABLE RESUMPTION POINT (as of wrap — re-judge against what changed):

• State: The pipeline is built, proven, committed. Alexander raw is clean-verified on CapableHands ~/capablehands/scratch/alexander-book{1,2,3}.md; book4 (The Luminous Ground) is page-marked in chamber-library canonical_texts/.../nature-of-order-vol-4-luminous-ground-alexander.md (just committed; the scratch book4 on CapableHands is the STALE pre-splice dirty copy — don't use it). All on main, pushed.
• First moves: (1) pull alexander-book1.md local (scp/ssh capablehands), run normalize_ocr.py dry-run to see how the mechanical layer handles it; (2) build its per-book structure.tsv — the hard part: Alexander's front matter reprints the WHOLE four-volume ToC, chapters are Roman-numbered (I., II.), page numbers restart per book, and "PART ONE/TWO" + "NOTES" are section-level — so parsing the real chapter list from the ToC needs care (and may need a richer map than Levi's flat TITLEanchor); (3) curatorial trim + §IV frontmatter; (4) verify + graduate. book4 is already local (page-marked) → normalize it too.

Other open horizons, ranked:

• Load-bearing (the thread): Alexander 1–4 graduation (+ book4). Then la-chute (EPUB, sectionless — different path; restored/clean, likely just needs frontmatter+verify+graduate).
• Owed follow-ons from today: item 1 tail — engine re-ingest so coverage-ledger.json + chunk-quality-report.md catch up to the re-pointed Levi (an engine run); item 2 — the CapableHands driver page-number note, BLOCKED on the steward granting Bash(ssh capablehands:*) permission (or approving a remote-write prompt "don't ask again").
• Governance: REVIEWED-45 owed from steward (chamber spec ratification, paper loop). The studium-engine §VI voice-manifest schema + the 84-file voice migration remain open (PENDING-42).
• Parked-with-reason: the ToC+offset auto-resolver (fragile; deferred until pattern forces it); re-OCR for page numbers (manual cheaper for current 5 books; preserve-page-numbers only forward); expanding Station I beyond the Gray Zone (steward's call).

PAUSE STATEMENT: I'm leaving at a clean milestone — the Chamber Library has a ratified constitution AND its first conformant text, with a reusable pipeline proven and everything committed/backed up across three repos. The graduation precedent (machine does the deterministic 95%, the human rules the irreducible 5%; every word provably faithful) is the thing I most want to find still pulling. On return I want to find whether the steward has granted the CapableHands write permission + wants the engine re-ingest, and then carry the precedent into Alexander.

LITERAL QUESTION for next-Claude: Does Alexander's gnarlier structure — the four-volume reprinted ToC, Roman-numeral chapters, per-book page restarts — work within the current per-book structure.tsv + normalizer, or does it need a richer instrument? And is the right unit one structure-map per volume, or one shared across the four-book series? (Sub-questions: did the steward grant Bash(ssh capablehands:*) so item 2 + Alexander pulls are friction-free? Does he want the engine re-ingest for the re-pointed Levi?)

State: chamber-library 7b46937 + studium-engine 38de1a9 + dotfiles adb3d88 all pushed; dotfiles ledger to commit at this wrap. Levi graduated + backed up. Alexander raw clean on CapableHands. ssh capablehands alias active (reads work; writes need the steward's permission rule).