dotfiles/claude/memory/project-mempalace-library-incremental-remine-strategy.md

name, description, type, originSessionId

name	description	type	originSessionId
MemPalace chamber-library re-mine — incremental strategy, never another multi-day loss	After 3 days lost to bge-m3+MPS chamber-library mine that produced unusable storage (chromadb format corruption + global embedding patch conflict), steward set the rule. Future library mining is incremental, prioritized by criticality, never a single multi-day pass.	project	45a40b1e-09cd-42bb-a41a-7298f78a8cf0

Context

The 2026-05-01 → 2026-05-03 chamber-library mine on bge-m3+MPS produced 933,576 drawers in storage but unreadable by chromadb 1.5.8 (format error: "Palace is NOT readable by chromadb 1.5.8"). Three days of CPU/MPS lost. Palace preserved at ~/.mempalace/palace.bge-m3-broken-2026-05-04/ for later forensic recovery.

Compounding architectural issue: the local patch in mempalace/backends/chroma.py (_mempalace_embedding_fn) forced bge-m3 (1024d) for ALL collections globally. This created a hard conflict with the pre-bge-m3 palace (mxbai 384d). Reverted 2026-05-04 to restore baseline functionality on the 131,418 pre-bge-m3 drawers (Obsidian vault, Claude transcripts, etc.). Patch saved at ~/_Dev/mempalace/.local-data/bge-m3-patch-2026-04-30.diff for reference.

Steward's rule (verbatim)

"go with path A. then we make a choice on how to remine the library incrementally beginning with the most critical texts. I'm not losing days to this again."

— David Glidden, 2026-05-04

The rule, expanded

1. No single multi-day mine of chamber-library ever again. A failure that's only visible after days of compute is unacceptable. Mines are incremental and validated end-to-end (storage AND search) before committing more compute.

2. Prioritize by criticality. The library has many texts; only some are load-bearing for active ARC work. Start with what's actually being used (recently cited / actively being read / referenced in current spec work).

3. Validate after each tranche. Storage write + search readback must both succeed before the next tranche starts. The bge-m3 mine's failure mode — storage succeeds, search corrupts silently — must be impossible going forward.

4. The architectural fix for the global-patch conflict is per-collection embedding configuration: bge-m3 for chamber_library (multilingual content), default mxbai for everything else. The current saved patch forces one global model and is not a viable long-term answer.

Open questions for the next library work session

1. Triage: which library texts are actually load-bearing right now? (probably: Bringhurst, Manutius the Younger, Berger, Athenaeus, Vico, Harrison, the Observer Problem sources, anything cited in active essays — but the steward should confirm)
2. Forensic on the broken palace: is the data recoverable, or do we re-mine source files? Cheapest first move: try mempalace migrate on palace.bge-m3-broken-2026-05-04/ (the format error is exactly what migrate exists for; it might just work). Status 2026-05-04: dry-run on big palace started but was killed when scope shifted to baseline restore.
3. Embedding strategy: continue with bge-m3 for multilingual quality, OR accept default mxbai for English-only criticality? bge-m3 quality non-negotiable per 2026-05-01 decision (chamber-library has critical French + some German texts), so the architectural fix (per-collection embedding) is the right path.
4. Per-collection embedding patch: refactor the saved patch in .local-data/bge-m3-patch-2026-04-30.diff to apply only when collection name matches chamber_library* (or via explicit allow-list). Should be ~10-line change. Test against a 100-file mine before any larger commitment.

Anti-patterns to refuse

• "Just re-run the mine, it'll probably work this time" — three days lost to that posture.
• "Skip the dry-run / verification step, we already know it works" — the bge-m3 mine wrote 933k drawers that look ready by mempalace_status and fail on first search. Storage success ≠ recall success.
• "It's only multilingual, English content might still work without bge-m3" — quality assessment was already made: chamber-library has critical French + German content. Don't relitigate.

Status 2026-05-04

Baseline restored. Library re-mine strategy parked until steward bandwidth + decision on triage criteria. Forensic on broken big palace is a candidate first move (cheap if it works).