session 2026-06-27: ocrmac conversion toolset calibrated + Making-corpus map + engine research; feedback-resurface-banked-notes memory

claude/memory/MEMORY.md

@@ -27,6 +27,7 @@ permalink: claude-memory/memory
 - [Skill-harvest register](skill-harvest-register.md) — **canonical home for proposed skills** awaiting steward authorization; the governed analog of PENDING.md for our own tooling. **Full review 2026-06-05**: `bmf-diagnose` + `/pre-build-audit` AUTHORIZED (build-on-need); `/spec-code-audit` DEFERRED (first L1 audit); `/arc-typeset` REJECTED (superseded by §I.k build enforcement); `/vignette` still deferred (Phase 2); Symmetria §3 flags + verification-ladder BUILT same evening.
 - [Verification ladder](reference-verification-ladder.md) — the named instruments (byte-identical compile gate, delta classification, censused-routes, fresh-clone gate, measure-toolchain-before-spec…); reach for the gate the claim's shape demands instead of re-deriving.
 - [Plane coordination workflow](reference-plane-coordination-workflow.md) — **NOW CENTRAL to the steward↔Seb workflow** (est. 2026-06-02): `app.plane.so/capablemind`, BMF project, thin effort-tasks `[BM #N]` over canonical GH issues (effort-only, no technical detail; labels/priority mirror GH; issue attached as link). Executor keeps task state current as the 6-hour relay. ⚠ BBF = BetterBridge *product*, NOT the board. Plane MCP user-scoped/OAuth; load tools via ToolSearch `select:mcp__plane__*`. `create_work_item` needs `description_html`.
+- [Resurface banked notes before re-deriving](feedback-resurface-banked-notes-before-rederiving.md) — when an idea was already captured (runbook known_gaps, skill-harvest, prior note), READ the note before re-deriving; re-derivation drifts (page-numbers mis-recalled as provenance vs structure-scaffolding, steward re-corrected 2026-06-27). The engine's own purpose, turned on my practice.
 - [Tool review after each use](feedback-tool-review-after-each-use.md) — steward directive (2026-06-16): review every tool we built after each run (success OR failure), capture what it taught, iterate until reliable; PASS-BUT-FALSELY is the priority signal. Log at `chamber-library/_curation/tool-evolution-log.md`. Proven same day (audit_cruft hardening → 160 hidden-dirty files surfaced).
 
 ## Canonical Workstream Trackers
@@ -43,7 +44,10 @@ permalink: claude-memory/memory
 - [Be (laundromat)](project-be-laundromat.md) — canonical workstream tracker established 2026-06-08 (Seb-relay of locked decisions). Be = Skemantix startup (Seb+David) funding CapableMind's funding-ladder; **bridge, not venture**. Decisions LOCKED: entity/exit (CapableMind decoupled, grant-funded), pricing (Living $12.99/mo · Archive $69.99/yr · Memorial $49.99/yr · Renovate ~$199 · $8.99 floor), CF Self-Serve Agency + versioned-template-package infra. **a11y gate MERGED (Pat 100/100/100).** Pre-revenue: the WTP gate = renovate Pat → charge her. **Discipline: stop adding spec until the gate clears → nothing for executor on be until then.** Repo @ `f43a0fd`.
 
 ## Active Session
-- [Session 2026-06-26 — Chamber spec ratified + OCR pipeline + Levi first graduation](session-2026-06-26-chamber-ocr-pipeline-levi-first-graduation.md) — **Chamber Library spec §§I–VI RATIFIED** (steward+jurist loop, run live): source-agnostic canonical form + work+chapter+quote citation (+sectionless reduction); **§V verbatim = faithful-under-*declared*-normalization, NOT byte-identity** (three-tier; resolved the spec's internal contradiction); **§VI two-sidecar Voice** (reading-index derived+hash-bound / voice-manifest authored engine-side; gate verifies ONLY vs canonical text; `voice:` tag stays in corpus). PENDING-42; **REVIEWED-45 owed from steward.** Sidecar-typology thinking → `studium-engine/docs/the-sidecar-typology-and-orchestration-2026-06-26.md`. **Built the reusable OCR→canonical pipeline:** `scripts/normalize_ocr.py` (drop pages · conservative seam-rejoin · dict-validated de-hyphenation [merge / auto-resolve-doubling / defer-compound] · **verbatim WORD-GUARD** §V · conversion record; raw off-canon §III; supersedes clean_ocrmac_pagination) + `scripts/insert_chapter_headings.py` (per-book TITLE⇥anchor structure-map, fails-loud). **Caught a PASS-BUT-FALSELY in my OWN guard** (passed 3 word-altering merges → dict-validate fix). **FIRST SPEC-CONFORMANT GRADUATION: Levi *The Drowned and the Saved*** (Rosenthal 1988) — §IV/§VI frontmatter, Coleridge epigraph, **10 verified chapter headings** (read body vs ToC), apparatus trimmed, verify_conversion PASS; raw→`converted_texts/` §III + conversion.yaml + structure.tsv; catalogue 1286. Steward: "crossed a milestone, set an excellent precedent." **Structure finding (load-bearing for Alexander):** olmOCR output has NO heading markup / NO body page numbers / PDFs NO outline → chapter structure recovered **curatorially from the ToC, per-book**; Alexander's 4 books share it (worse: 4-vol reprinted ToC, Roman numerals, per-book page restarts). Steward's page-number idea **tested→doesn't-stick** for current files (furniture already stripped at OCR) → banked as a **driver enhancement** (runbook known_gaps.next). **Committed+pushed all 3 repos:** chamber `7b46937` (→skemantix), dotfiles `adb3d88`, studium `6abb593`+`38de1a9`. **Engine re-pointed** `levi-gray-zone` omnibus→standalone (manifest+sidecar; Gray Zone now clean ## ch L116–257). **PULLING THREAD: apply the proven pipeline to the rest of the OCR sources — Alexander Books 1–4 next** (book4 page-marked in chamber; books 1-3 raw clean on CapableHands `~/capablehands/scratch/`). **Owed:** engine re-ingest (coverage-ledger/chunk-quality stale to re-pointed Levi); CapableHands driver note BLOCKED on steward granting `Bash(ssh capablehands:*)`. `ssh capablehands` alias active. **Q: does Alexander's gnarlier ToC fit the current per-book structure-map, or need a richer instrument (one map per vol vs shared)?**
+- [Session 2026-06-27 — ocrmac conversion toolset CALIBRATED + Making-corpus advance + engine research](session-2026-06-27-ocr-toolset-calibrated-making-corpus-research.md) — Marathon. Alexander book1 graduation → **discovered olmOCR SCRAMBLES 2-column figure-dense books** (reads across the gutter; the verbatim word-guard can't catch it — PASS-BUT-FALSELY at the corpus level) → **built+calibrated the ocrmac (Apple Vision) column-aware conversion toolset**: (1) **column detection** = line-crossing-of-narrow-lines + line-width (DPI was the key, 120→150); validated all 491 pages; (2) **de-hyphenation** = **wordfreq, MULTILINGUAL** `--lang en|fr|de|es` (merged-Zipf≥3.4→merge / both-parts≥4→compound / unknown-lowercase→**merge_flag** catches OCR+reflow errors); Camus/Handke forward-req met by design; (3) **structure recovery** = steward's page-number method (scaffolding NOT provenance; retain→map ToC printed-page→scan→place→strip) **PROVEN 9/9**. Tools evolved: `scripts/normalize_ocr.py` (+within-page rejoin, page-top runhead detection, `--no-heading-recovery`, wordfreq), `scripts/insert_chapter_headings.py` (2-level+block-replace; Levi byte-identical). Method `docs/ocr-conversion-method-2026-06-27.md`. **Then compute-deployed (steward: spend the compute, except L1):** engine-design **research sweep DONE** (`studium-engine/docs/research-grounded-reasoning-bounded-corpus-2026-06-27.md`; 107 agents/23 verified — citation **correctness≠faithfulness** [57% post-rationalized] vindicates "boundedness=trust"→couple attribution to generation; adopt **SelfCite+eTracer+MiniCheck**; engine's genealogy/temporal + multi-voice signature capabilities are GREENFIELD = 2 sweeps owed); **Making-corpus readiness map** (`chamber-library/docs/making-corpus-conversion-readiness-2026-06-27.md`; 30 raw, mostly EPUB=fast); **Musil English EPUB converted** (steward upload; 464k words; structure scoped via NCX anchors). **Steward corrections:** page-numbers=scaffolding-not-provenance (→`feedback-resurface-banked-notes-before-rederiving`); Levi drop-caps RECOVERED (re-raised wrongly 3×) but a **within-page-rejoin hyphenation final pass owed** ("essentially ready"); p42-is-2col (misread the image). **PULLING THREAD: finish the conversion toolset (package driver + furniture layer) + build the Making corpus — Position I (Musil structure recovery + Levi hyphenation pass) unblocks the engine's first pattern-finder pass.** UNCOMMITTED for steward review: chamber scripts+docs, studium research doc (_scratch/ + staged olmOCR Alexander raw are SUPERSEDED — don't commit). CapableHands: ocrmac installed, /tmp cache. ssh/scp capablehands persisted. **Q: unify ToC-page-number (OCR) + NCX-anchor (EPUB) structure recovery into one format-agnostic "structure-from-authoritative-ToC" step?**
+
+## Archived (2026-06-27 — ocrmac toolset calibrated + Making-corpus + engine research)
+- [Session 2026-06-26 — Chamber spec ratified + OCR pipeline + Levi first graduation](session-2026-06-26-chamber-ocr-pipeline-levi-first-graduation.md) — Chamber Library spec §§I–VI RATIFIED (source-agnostic canonical form; verbatim=faithful-under-declared-normalization; two-sidecar Voice); built the OCR→canonical pipeline + FIRST spec-conformant graduation (Levi). PENDING-42 / REVIEWED-45-owed. [Superseded re: OCR — olmOCR pipeline replaced by ocrmac column-aware toolset 2026-06-27.]
 
 ## Archived (2026-06-26 — Chamber OCR pipeline + Levi first graduation)
 - [Session 2026-06-24→25 — OCR workhorse proven → the Chamber Library needs a spec](session-2026-06-25-chamber-library-spec-born-from-ocr-workhorse.md) — **Stood up CapableHands (M4 64GB, `david@10.0.1.136`) as an isolated MLX/olmOCR workhorse** (uv+py3.12+mlx-vlm; model `alexgusevski/olmOCR-7B-0225-preview-q8-mlx`; isolated from Peter's `capablemind` Ollama). **Levi** (*Drowned & Saved*, Rosenthal 1988 — drop-cap gate CLEARED) + **Alexander Book 4** OCR'd; **caught a PASS-BUT-FALSELY** (driver dumped JSON on no-text pages + ToC dot-loops) → patched driver (null→marker, `repetition_penalty`, `--pages`) + `splice_pages.py` → both clean. **2 ARC posts published** (The Allée / Two Notes, Aix; source-commit was a loose end → committed at wrap). **THE REAL ARC:** a citation-convention question → **steward challenged my two-tier (PDF=page / EPUB=chapter) framing** → resolved to **ONE source-agnostic canonical standard** (clean prose + intrinsic `##`/`---` structure + citation by *work+chapter+verbatim quote*; no page markers; page-provenance off-canon) → **realization that the chamber library is a substrate needing its own SPEC.** **Seed drafted:** `chamber-library/docs/chamber-library-specification.md` (`[PROPOSAL]`, charter-coupled; §§II-III settled, **§VI Voice RESERVED/undesigned** per steward, §V/§VII/§IX open). **Careless-edit mistake recovered:** stripped *La Chute*'s 5 structural `\`-break spacers as "stray" (steward caught: 6 evenings) → restored from pristine backup. **PULLING THREAD: ratify spec §§II-III → build OCR+EPUB normalizers (→ one form) → conformant graduation** (held until then; Books 1-3 OCR-ing background PID 2653). **Q: do steward+jurist ratify §§II-III?** [[project-arc-open-work-register]] §G (Gwern: link-rot G1 / progress-bar G2 / Gwern-audit G3). CapableHands password steward-supplied → **rotate.**