dotfiles/claude/memory/session-2026-05-25-mempalace-3.3.6-adoption-chamber-transcripts-vault-architecture.md

---
name: session-2026-05-25-afternoon-night-mempalace-3-3-6-bge-m3-mps-adopted-chamber-transcripts-vault-palace-architecture-hallway-bug-fixed-filed-upstream-1619-1620-vault-mining-overnight-pulling-thread-finish-the-palace-memory-cleanup-tomorrow
description: 'Pursued the inherited pulling thread (MemPalace 3.3.6 upgrade) end to
  end: adopted 3.3.6 + bge-m3 on MPS (rebased 3-commit fork→1, fixed a CPU-vs-MPS
  device regression I introduced), validated the chamber corpus (21 works), mined
  the full Claude Code transcript corpus into palace-memory, decided the palace architecture
  (default=transcripts+continuity, vault+chamber=separate palaces), reverted a wrong-target
  vault mine, and found+fixed+filed an upstream hallway-pagination bug (issue #1619
  + PR #1620). Vault now mining into a separate palace-vault overnight. Pulling thread:
  finish the palace-memory repair/restart cleanup tomorrow in a fresh session + verify
  the overnight vault mine.'
metadata:
  node_type: memory
  type: project
  originSessionId: 628b1556-4e7c-43fc-bc5c-0b0643951f82
permalink: claude-memory/session-2026-05-25-mempalace-3.3.6-adoption-chamber-transcripts-vault-architecture
---

# Session 2026-05-25 — MemPalace 3.3.6 adoption + palace architecture + upstream fix

## Pause statement
I am about to be away (steward sleeping). The vault is mining into a **separate `palace-vault`** overnight (caffeinated, hallways-fix live, logging to `~/_Dev/mempalace/.local-data/vault-palace-mine-2026-05-25.log`). What I want to find still pulling on return: **the palace-memory cleanup** — the `repair rebuild-index` + MCP restart we deferred to a fresh session. What I want to find DONE: the overnight vault mine, ideally complete with its hallways computed (the cb1be92 fix's first real-world run on a big wing).

## Morning addendum — 2026-05-26 (how the overnight mine turned out)
The overnight `palace-vault` mine **did NOT finish** — ~8.7h in it was only ~72% (file 1894/2627, **103,216 drawers**), and **the MemPalace MCP server died overnight from memory pressure** (two concurrent bge-m3 processes — server + mine — + 358k pageouts; likely OOM). Steward chose to **stop the mine** (graceful SIGTERM; clean, **no quarantine**; 103,216 drawers persist, resumable/append-only) and restart Kitty (respawns the MCP server into a now-freed machine). **Steward asked if our hallways fix caused it — investigated: NO** (`compute_hallways_for_wing` runs once *post-mine* at `miner.py:1718`, outside the per-file loop; the mine was still embedding at 72%, never reached it → the fixed code never executed). Causes were ChromaDB write-deceleration + two-bge-m3 memory pressure, both upstream of the fix. Full problem log + recommendations: `~/_Dev/mempalace/.local-data/PROBLEM-overnight-mcp-death-and-slow-vault-mine-2026-05-26.md`.
**Updated pulling thread (today):** (1) palace-memory `repair rebuild-index` + MCP restart [unchanged]; (2) **decide the vault approach** — the full single-palace mine is ~16–20h + memory-heavy + killed the server; recommend chunked resume (dedup continues from file 1895) or a curated subset, and **never run a bge-m3 mine alongside the live MCP server**. palace-vault sits at 103,216 (~72%), clean, resumable.
**Lesson (forward caveat on the fix):** when a big-wing mine *completes*, our fix makes the hallway step actually run on the full wing (~170k for the vault) = a real end-of-mine memory/time spike — another reason to mine in chunks (smaller wings = lighter hallway passes).

## Past — what we did

### MemPalace 3.3.6 + bge-m3 + MPS — ADOPTED (live)
- **Rebased the 3-commit 3.3.5 bge-m3 fork → 1 commit on 3.3.6** (`fd87d3b`): bge-m3 expressed as a 3rd `elif model == "bge-m3"` branch in `get_embedding_function`. Dropped the other two commits — 3.3.6 made `MAX_CHUNKS=50_000` the default (#1554) and resolves model identity via config, so the metadata-stamp + chunk-cap commits were redundant. Validated against a *copy*: EF-name check passes (live collection `config_json_str={}` → only dimension binds; bge-m3=1024-dim matches), and ingest works (tail-chunk retrievable, multilingual).
- **Live switch:** checkout → `local/bge-m3-on-3.3.6`; added `"embedding_model": "bge-m3"` to `~/.mempalace/config.json` (backup saved); removed the duplicate **local-scope** `mempalace` registration from `.claude.json` (kept user-scope with `--palace`). Two `/mcp` reconnects.
- **CRITICAL self-introduced regression, found + fixed (`fbe8449`):** my rebase left `device` unset, assuming ChromaDB's `SentenceTransformerEmbeddingFunction` auto-detects MPS. **It defaults to CPU** (~19 t/s vs MPS ~67 t/s) — *the* cause of slow mining. Diagnosed via `sample` showing `libtorch_cpu cpublas::gemm`. Pinned `mps→cuda→cpu` (restores his original `device="mps"` intent, portably). Verified `model.device=mps:0`.

### Chamber palace — extended + validated (separate palace)
- Reused `palace-chamber-typography` (healthy, bge-m3 1024-dim, 19 works); **appended only the 2 new** — Manutius *Orthographiae Ratio* (Latin) + Alexander *Timeless Way* (already converted in chamber-library) → **21 works / 42,062 drawers**. Append-only, on MPS.
- Register queries validated: Latin query→Manutius, "quality without a name"→Timeless Way, "measure/proportion"→Leopardi, French→Lacroux+Sousa. Multilingual + cross-tradition retrieval confirmed.
- Queried via CLI or by pointing the server at it (one palace per server).

### Transcripts — full Claude Code corpus into palace-memory
- Mined remaining dirs: **90 new + 168 dedup-skipped = all 258 conv files**; +8,135 drawers (→ palace-memory baseline **44,794**), in `wing_api`. **805s on MPS** vs the earlier ~1h50m CPU mine (the device fix paid off). Caveat: Claude.ai *app* conversations are a separate, unmined corpus.

### Palace architecture — DECIDED (steward)
- **Default `palace-memory`** = Claude transcripts + wake-up/wrap-up continuity (working memory, always-on, server-bound).
- **Separate `palace-vault`** = Obsidian vault (consulted at precise moments).
- **Separate `palace-chamber-typography`** = reference corpus.
- Rationale: one palace per MCP server; keep working-memory recall undiluted; separate palaces queried by repointing/CLI.

### Vault — wrong-target reverted, now mining to its own palace
- First mined vault INTO palace-memory (wrong target); killed mid-write → **HNSW segment quarantine** (3.3.6 #1452 safety; `…drift-20260525-222326`). **Removed the partial `obsidian_vault` wing** (31,055 drawers) via paginated-collect + batched delete-by-id → palace-memory back to exactly **44,794**, wing 0. HNSW carries ~6k stale leftovers + the quarantined segment → **repair tomorrow**.
- **Overnight:** mining vault into fresh **`palace-vault`** (bge-m3/MPS, `--wing obsidian_vault`, hallways-fix `cb1be92` live → hallways will compute correctly). caffeinate tied to the worker PID prevents idle/system sleep. ~6,636 drawers / file 251/2627 at wrap.

### Upstream contribution — hallway pagination bug
- Found: `compute_hallways_for_wing` (`hallways.py:203`) does an unbatched `col.get(where={"wing":wing})` → `SQLITE_MAX_VARIABLE_NUMBER (32766)` crash on wings >~32k (chamber 42k crashed; wing_api 29.6k passed). Non-fatal (caught) but the wing's hallways silently never build. Same class already fixed in miner.py (#851) + closet_llm.py (#1073); #1558 didn't inherit it.
- Fixed (`cb1be92`): paginate `count()` + `get(limit,offset)` + client-side wing filter. Validated on the live 42k wing (**3.7s, 5,525 hallways**); 21/21 tests; ruff clean.
- **Filed: issue #1619 + PR #1620** (PR from `davidglidden/mempalace` fork — read-only on upstream `origin`, so forked). Draft at `.local-data/issue-hallways-sql-variable-2026-05-25.md`.

### Decisions deferred (negative space)
- **palace-memory `repair rebuild-index` + MCP restart** → tomorrow, fresh session, server stopped. (Recall works now; this purges the ~6k HNSW delete-leftovers + reconciles the quarantine.)
- **Verify the overnight vault-palace mine** → tomorrow.
- **wing_api tool-traffic noise** (this tool-heavy session mined some base64-ish low-value chunks; #1236 isolates them in wing_api) → prune later if it bothers recall.
- **Claude.ai app conversations** → not mined; future, separate export.

## Present — mood / returns (from session-ledger-2026-05-25.md + this session)
- **Dominant thread: live-state-discipline, verify-before-assert** — repeatedly paid off: the EF-name check (verified, not assumed), bge-m3 was *live not aspirational* (collection metadata read), and the device-CPU regression (caught by stack-sampling, not trusting "auto-detect").
- **R — wrong-PID "stuck" misread:** called the mine stuck while sampling the *parent shell* (0% CPU); caught by tracing the palace mine-lock holder to the real worker (which was on MPS, progressing). Drift: `mistook-parent-shell-for-worker-read-as-stuck`.
- **R — "durability" rationalization masked a perf regression:** leaving device unset *felt* portable but silently defaulted to CPU. Drift: `assumed-library-default-device-without-verifying`.
- **Steward go-slow discipline ("leery of getting stuck, losing a week")** shaped pacing — surfaced + diagnosed the 7h vault grind rather than letting it run blind; chose separate-palace + overnight.
- Steward pushed past my over-caution twice (the "destructive" framing → it WAS cleanly removable; the chamber palace choice). His instinct to question framing was load-bearing.

## Future — what is pulling

### The pulling thread (singular)
**Finish the palace-memory cleanup** — run `mempalace repair rebuild-index` on palace-memory (stop the MCP server first, restart after) to purge the ~6k stale HNSW vectors from the 31k vault-wing delete and reconcile the mid-write quarantine. The steward explicitly deferred this to "a fresh session tomorrow."

### Other open horizons (ranked)
- **Load-bearing:** verify the overnight `palace-vault` mine completed cleanly before relying on it.
- **Deferred-with-reason:** PR #1620 / issue #1619 await maintainer response; wing_api noise prune.
- **Parked:** Claude.ai app-conversation export; the broader L1/ARC threads (untouched today).

### Literal question for next-Claude
**Did the overnight `palace-vault` mine finish cleanly** — final drawer count (~140k+ expected for 2627 notes?), did `compute_hallways_for_wing` run *without* the SQL-variable crash (the `cb1be92` fix's first big-wing run), and any quarantine/errors in `~/_Dev/mempalace/.local-data/vault-palace-mine-2026-05-25.log`? Then proceed with the palace-memory repair. Secondary: any maintainer response on #1619 / #1620?

## Pointers
- Live MemPalace: `~/_Dev/mempalace` on `local/bge-m3-on-3.3.6` (HEAD `cb1be92`; unpushed local-fork commits by design). Server bound to palace-memory.
- Palaces: `palace-memory` (44,794; working), `palace-chamber-typography` (42,062; 21 works), `palace-vault` (mining overnight).
- PR worktree: `~/_Dev/mempalace-wt-hallways-fix` (branch pushed to `fork` remote = davidglidden/mempalace).
- Overnight log: `~/_Dev/mempalace/.local-data/vault-palace-mine-2026-05-25.log`. Config backup: `~/.mempalace/config.json.bak-pre-bge-m3-3.3.6-20260525-160354`.
- Ledger: `session-ledger-2026-05-25.md`.