dotfiles/claude/memory/project-chamber-typography-mining-plan-2026-05-15.md

---
name: Chamber Typography Mining Plan — 2026-05-15 → next session
description: Detailed execution plan for mining a priority voice subset from chamber-library
  into a clean palace, enabling sustained corpus consultation for ARC work in flight
  (multilingual typography spec including new English §3, AldineXXI page rewrite,
  attention page rewrite, Print posture refining). Self-contained for either /wrap-up
  + /wake-up or clear-context execution.
type: project
originSessionId: a5214e9b-ee20-435a-8333-f7e645f8ede0
permalink: claude-memory/project-chamber-typography-mining-plan-2026-05-15
---

# Chamber Typography Mining Plan — 2026-05-15

## Why this plan exists

Today's session diagnosed the death of the chamber-library bge-m3 mine: 1,290,916 drawers in SQLite, HNSW only 27% indexed (346,390), 37 corruption snapshots accumulated. Root cause: HNSW `batch_size:50000` + `sync_threshold:50000` set in the collection config, combined with bge-m3 on MPS being too slow to flush 50k-vector batches before the next mega-write arrived. Lock fix #1162 alone did not prevent it.

The steward's ARC work in flight needs corpus consultation: multilingual typography spec refinement (FR §1 + ES §2 v0.2; new English §3 to be added), AldineXXI page rewrite (8-voice chavruta from 2026-05-10), attention page rewrite, Print posture refining. All sustained-consultation-shaped, not one-shot lookup work. Direct file reads would tax out by the third lookup.

The decision: mine a **critical voice subset** (~15–20 source files; ~50–100k drawers) into a **separate clean palace**, with HNSW config fixed at creation. Vault gets its own palace alongside. Existing broken chamber palace moves aside (preserved as Curiosity Engine Stage 1 substrate candidate if Jurist authorizes).

## Inherited state (read before executing)

- **MemPalace branch**: `~/_Dev/mempalace/` on `local/bge-m3-on-3.3.5`. Three local commits: bge-m3 swap, embedding_model stamping, MAX_CHUNKS_PER_FILE 500→50_000. Branch stays; only HNSW config changes.
- **Broken chamber palace**: `~/.mempalace/palace/` — 1.29M drawers, HNSW broken, 11 `.corrupt-*` + 26 `.drift-*` snapshots. DO NOT TOUCH except to move aside.
- **Hooks state**: DISABLED at `~/.claude/settings.json`. Backup at `~/.claude/settings.json.backup-2026-05-14-pre-mine-completion`. Restore ONLY AFTER pointing at the new vault palace (NOT the broken chamber palace).
- **Curiosity Engine**: separate project, repo at `github.com/davidglidden/curiosity-engine` (private), seed brief committed (`e13cfad`), awaiting Jurist review. The chamber-typography subset mined per THIS plan serves a dual role: immediate ARC consultation AND eventual Stage 1 test substrate for Curiosity Engine.
- **Chamber-cruft state**: per `project-chamber-cruft-restoration.md` — 399 files total: 162 clean / 49 near-clean / 124 CLEANUP / 62 severe RECONVERT. Priority voices for this plan are unknown-state until audited (Phase 1b below). Bachelard explicitly named as severe-RECONVERT in 5 books.

## Phase 1 — Diagnostics (mandatory before any mining)

### 1a. Decouple `MAX_CHUNKS_PER_FILE` from HNSW `batch_size` (~15 min)

**Important context (steward correction 2026-05-15)**: `MAX_CHUNKS_PER_FILE = 50_000` is DELIBERATE — local commit `604df3b` raised it from 500 because Loeb bilingual files (long scholarly bilingual editions) were being silently truncated/skipped at the lower limit. **That dial must stay at 50_000.** Lowering it would re-break Loeb ingestion.

The corruption-causing dial is the HNSW `batch_size:50000` in `collections.config_json_str`, which appears to share the number (probably coupled to MAX_CHUNKS_PER_FILE in code). These need to be **decoupled**, not both lowered.

Goal of the investigation: find where HNSW `batch_size` and `sync_threshold` are set when a collection is created, identify if/how they're coupled to MAX_CHUNKS_PER_FILE, and decouple them. Target values: HNSW `batch_size=1000`, `sync_threshold=1000` (sane for bge-m3 MPS throughput) while MAX_CHUNKS_PER_FILE stays at 50_000.

Search pattern:
```
cd ~/_Dev/mempalace
grep -rn "batch_size" mempalace/ | grep -iE "50000|50_000|MAX_CHUNKS|hnsw"
grep -rn "sync_threshold" mempalace/
grep -rn "MAX_CHUNKS_PER_FILE" mempalace/
grep -rn "hnsw" mempalace/ | grep -iE "config|batch|sync|create"
```

Two plausible coupling shapes:
- (a) Collection-creation code sets `hnsw.batch_size = MAX_CHUNKS_PER_FILE` directly (single-source coupling). Fix: decouple — set HNSW batch_size from its own constant (e.g., `HNSW_BATCH_SIZE = 1000`).
- (b) Separate but happens to share the value. Fix: just lower HNSW batch_size in whichever file sets it.

Fix path: env var override at mine time if possible (least invasive). If env var doesn't exist, one-line local commit decoupling the constants.

Validation: after fix, create a tiny test palace and inspect `collections.config_json_str` — should show `hnsw.batch_size:1000` AND mining a long Loeb file should still produce >500 chunks (i.e., MAX_CHUNKS_PER_FILE still effectively 50_000).

### 1b. Cruft audit for priority voices (~15 min)

Cross-reference the priority voice list (Phase 4 below) against `project-chamber-cruft-restoration.md`. For each priority voice, determine: CLEAN / NEAR-CLEAN / CLEANUP / RECONVERT.

Specifically check the load-bearing voices not used in yesterday's sessions (which had to be clean to be useful):

| Voice | Books | Used recently | Cruft state |
|---|---|---|---|
| Bringhurst | Elements of Typographic Style | yes (AldineXXI) | likely clean |
| Hochuli | Detail in Typography | yes | likely clean |
| Tschichold | Form of the Book, New Typography | yes | unknown |
| Lacroux | Orthotypographie v1, v2 | yes (multilingual spec) | clean (operative yesterday) |
| Lexique Imprimerie nationale | — | yes | clean (operative yesterday) |
| Sousa | Ortografía y ortotipografía | yes | clean (operative yesterday) |
| Butterick | Practical Typography | — | unknown |
| Hart's Rules | — | — | unknown |
| Chicago Manual of Style | — | — | may not be in library |
| Lupton | Thinking with Type | yes (chavruta) | unknown |
| Manutius / Margolis | — | yes (chavruta) | unknown |
| Alexander | Timeless Way, Pattern Language | yes (chavruta) | unknown |
| Bachelard | Poetics of Space, Earth and Reveries | yes (chavruta) | **RECONVERT** (5 books flagged severe) |
| Calvino | Six Memos | yes (chavruta) | unknown |
| Müller-Brockmann | Grid Systems | — | unknown |
| de Hamel | — | — | unknown |
| Saenger | Space Between Words | — | unknown |
| Williamson | — | — | unknown |
| Davies | — | — | unknown |

### 1c. Cruft audit confirms cleaning targets (decisions absorbed 2026-05-15)

**Scope decision (steward, 2026-05-15)**: option (b) — clean one representative book per foundational voice in the cruft bucket. Minimum-viable-clean per voice; not the full chamber-library presence per voice.

Foundational voices to be cleaned, with per-voice book choice locked:
- **Alexander** — *Pattern Language* (severe RECONVERT)
- **Bachelard** — *Poetics of Space* (severe RECONVERT, multiple Bachelard books flagged)
- **Heidegger** — *Poetry, Language, Thought* (severe RECONVERT)

Foundational voices to be audited (state unknown; clean if needed):
- **Berger** — *Ways of Seeing*
- **Sennett** — *The Craftsman*
- **Vico** — *New Science* (or selected passages)
- **Leopardi** — *Zibaldone* (full text is ~2,500 pages; selected portions likely)
- **Harrison** — *The Dominion of the Dead* (likely clean — ARC uses Harrison heavily)

Any of these found in CLEANUP / RECONVERT bucket during audit gets the same Alexandrian per-file treatment (Phase 2). If clean as-is, skip to mining.

### 1d. Attention page voices (decisions absorbed 2026-05-15)

Confirmed by steward:
- **Carruthers** — *The Book of Memory* — memory + attention + medieval rhetoric
- **Weil** — *Gravity and Grace* — attention as moral act

Two voices. Sufficient for chavruta-shaped tension.

## Phase 2 — Cleaning of foundational voices (~3–5 days, parallelisable across short sessions)

Per-voice cleaning per the 1c decision. For each foundational voice in CLEANUP / RECONVERT bucket, produce a clean version of the chosen representative book using:

- `~/_Dev/chamber-library/scripts/repair_epub_headings.py` (heading promotion via TOC) — for scholarly EPUBs
- `pandoc -t gfm-raw_html` (suppress HTML residue) — standard EPUB pipeline
- Alexandrian per-file craft standard — verify chapter integrity, footnote placement, no cruft tokens in body, proper section structure
- Filing discipline: EPUBs stay in `~/Documents/__Chamber Source EPUBs/`; cleaned MDs land in `~/_Dev/chamber-library/converted_texts/`

Per-book estimated effort (varies by source quality):
- *Pattern Language* (Alexander) — known severe; ~half-day
- *Poetics of Space* (Bachelard) — known severe; ~half-day
- *Poetry, Language, Thought* (Heidegger) — known severe; ~half-day
- Berger / Sennett / Vico / Leopardi / Harrison — audit first; clean as needed (variable, ~half-day each if needed)

**Discipline reminders during cleaning** (from previous chamber-cruft work):
- Each file gets its own attention; no batch automation
- Verify source-file integrity by inspection, not by tool output alone (per `feedback-pandoc-epub-suppress-raw-html.md`)
- If a cleaning attempt fails (source too damaged, OCR errors, scanned-image content), surface that — don't paper over

**Stop condition for Phase 2**: all 8 foundational voices represented by one clean book + any typography masters or multilingual / attention voices found needing cleaning during Phase 1b audit.

**Parallelisation note**: cleaning is independent per-file. Multiple short sessions can advance Phase 2 without coordination overhead. Phases 3+ wait for Phase 2 to fully complete.

---

## Phase 3 — Setup (~15 min total)

### 3a. Move broken chamber palace aside

```
mv ~/.mempalace/palace ~/.mempalace/palace.broken-chamber-2026-05
```

Preserves the SQLite of 1.29M drawers. If Curiosity Engine ever wants to test `repair --mode from-sqlite` on this substrate, the data is available. Steward's call to reclaim disk space later.

### 3b. Hooks restoration pointed at vault palace

```
cp ~/.claude/settings.json.backup-2026-05-14-pre-mine-completion ~/.claude/settings.json
```

Then edit settings.json to set `MEMPALACE_PALACE_PATH` (or equivalent env var) for the hook invocations to `~/.mempalace/palace-vault/`. CRITICAL: do NOT let hooks fire against the broken chamber palace or against an empty default.

Alternative: rename whichever palace is to be the default to `~/.mempalace/palace/`. The vault palace is the right default since it's where session/diary writes go.

### 3c. Apply Phase 1a config fix

Per Phase 1a finding — env var or local commit. Verify with a tiny test palace creation that `config_json_str` shows sane values.

## Phase 4 — Vault re-mine (~15–30 min)

Vault path: `/Users/davidglidden/Library/Mobile Documents/iCloud~md~obsidian/Documents/David, root-and-branch`
Git mirror: `/Users/davidglidden/_Dev/david-root-and-branch-vault-git`

Use git mirror to avoid iCloud sync interference during mine. Mining command roughly:
```
mempalace --palace ~/.mempalace/palace-vault mine ~/_Dev/david-root-and-branch-vault-git --mode convos --wing vault
```
(verify exact flags against `mempalace mine --help` at execution time)

Validation:
- `mempalace --palace ~/.mempalace/palace-vault status` shows expected drawer count
- Recall test on a known-recent note returns it

## Phase 5 — Chamber-typography subset mine (~1–2 hours wall-clock; runs in background)

Mine the 20-book locked corpus into `~/.mempalace/palace-chamber-typography/`.

**The 8 foundational voices (per ARC colophon Lineage, all must be present)**:
- Alexander — *Pattern Language* (cleaned in Phase 2)
- Bachelard — *Poetics of Space* (cleaned in Phase 2)
- Berger — *Ways of Seeing* (audit; clean if needed)
- Sennett — *The Craftsman* (audit; clean if needed)
- Vico — *New Science* or selected (audit; clean if needed)
- Leopardi — *Zibaldone* selected portions (audit; clean if needed)
- Harrison — *The Dominion of the Dead* (audit; likely clean)
- Heidegger — *Poetry, Language, Thought* (cleaned in Phase 2)

**Typography masters (5 books, for AldineXXI / Print posture / cross-task)**:
- Bringhurst — *Elements of Typographic Style*
- Tschichold — *Form of the Book*
- Hochuli — *Detail in Typography*
- Lupton — *Thinking with Type*
- Calvino — *Six Memos* (literary register for AldineXXI; distinct contribution from Bachelard)

**Multilingual authorities (5 books)**:
- Lacroux — *Orthotypographie* v1
- Lacroux — *Orthotypographie* v2
- *Lexique de l'Imprimerie nationale*
- Sousa — *Ortografía y ortotipografía del español*
- Butterick — *Practical Typography* (for new English §3)

**Attention voices (2 books)**:
- Carruthers — *The Book of Memory*
- Weil — *Gravity and Grace*

**Total: 20 books.** Cut from earlier lists: Manutius (covered by Bringhurst/Tschichold/Hochuli's treatment of typography history); Hart's Rules (Butterick + Bringhurst cover modern English); Chicago Manual (encyclopedic, overkill for our scope); Müller-Brockmann/de Hamel/Saenger/Williamson/Davies (typography breadth; defer — not blocking ARC-up-to-vignette work).

Voice metadata: each source mined with `voice` (e.g., "Bachelard") and `work` (e.g., "Poetics of Space") fields in frontmatter or per-source metadata so `--source` or `--wing` scoping works. Verify mempalace's source-level scoping at execution.

Approximate scale: 20 books × avg 3–5k drawers each (with sensible chunking; no Loeb files in this subset so MAX_CHUNKS_PER_FILE will not actually be hit) = 60–100k drawers. At bge-m3 MPS ~50k/h with FIXED HNSW config (per Phase 1a): 1–2 hours wall-clock.

## Phase 6 — Validation (~15 min)

Three test queries:

1. **Cross-language**: `"italic for emphasis"` — should surface Lexique (FR) AND Bringhurst (EN) within top 10 results. Cross-language retrieval was confirmed at cosine 0.64 on 2026-05-11 PR #442 test.
2. **Specific concept**: `"Van de Graaf canon"` or `"type-area construction"` — should surface Hochuli, Bringhurst, Tschichold; one or two strong hits per source, not 25 from one book.
3. **Cross-voice synthesis**: `"voice and form"` or `"reading as construction"` — should surface multiple voices (Bachelard, Carruthers if present, Bringhurst, etc.), not one source dominating.

Failure modes to watch:
- Density-dilution returning (one source dominates 70%+ of hits)
- Empty results on terms that should be present
- High-cosine matches on cruft tokens (should not appear if Phase 1c done properly)

If validation fails: DIAGNOSE, do not rationalize. The failure is information.

## Phase 7 — ARC work begins

With the chamber-typography palace consulted on demand:
```
mempalace --palace ~/.mempalace/palace-chamber-typography search "query" --source bringhurst
```
(verify flag syntax against actual CLI)

Suggested task order:
1. **Print posture refining** — already lived-with for ~24h; iterating on feel. Light first task.
2. **Multilingual typography spec refinement** — FR §1 + ES §2 to v1.0 LOCKED + new English §3 drafted to v0.1 then v0.2. ~3–5h sustained.
3. **AldineXXI page rewrite resumption** — masters' reading session from 2026-05-10 pulling thread.
4. **Attention page rewrite** — after voices named.

## Risks (carry forward as discipline)

- **Naive mining**: Phase 3 or 4 without Phase 1 = repeat corruption cycle. The discipline is: investigation before action.
- **Hook restoration to wrong palace**: hooks pointing at the broken chamber palace would write diary entries into a dead substrate. Verify hook palace path before restoring.
- **Voice list overreach**: 15–20 sources is sufficient. Resist the urge to "mine everything that might be useful." More is worse here — density-dilution risk grows.
- **Cruft contamination**: mining a source still in CLEANUP/RECONVERT will reproduce density-dilution for that voice. Either clean or skip; don't mine dirty.
- **Curiosity Engine context drift**: this subset palace has a dual role. Don't lose sight of (a) immediate ARC consultation, (b) Curiosity Engine Stage 1 test substrate.

## Success criteria

- Clean palace at `~/.mempalace/palace-chamber-typography/`
- Voice-scoped recall works
- Density-dilution not visible at this scale
- ARC tasks (multilingual spec including new English §3, AldineXXI, attention, Print posture) can proceed with corpus consultation
- Vault palace also clean and hook-fed
- Broken chamber palace preserved (not deleted) for Curiosity Engine consideration

## What this plan does NOT cover

- Curiosity Engine implementation (gated on Jurist review of seed brief)
- Deep landscape audit (Jurist's territory per seed brief §11.7)
- Full chamber-library cleanup (multi-day; separate workflow)
- L1 work (check `gh pr view 172` / `gh issue view 165` / `gh issue view 167` in transition; pick up only if Seb has engaged)

## Discipline notes for the executor in the next session

- **Symmetria init** at session start if non-trivial (this plan is non-trivial; do it).
- **Drift pattern to watch**: *warning-in-context-still-launched* — Phase 1 is the warning; do not skip to Phase 3 because mining feels productive. Investigation is productive too.
- **Drift pattern to watch**: *forcing-the-plan-onto-changed-evidence* — if Phase 1b finds the cruft state is much worse than expected, the plan may need restructuring. Surface, don't paper over.
- **Honest empty**: if a phase produces a finding that invalidates a downstream phase, say so and stop. The steward authorizes restructure; the executor doesn't barrel through.