57 lines
4.1 KiB
Markdown
57 lines
4.1 KiB
Markdown
---
|
|
name: MemPalace bge-m3 mine — drawers stored, search broken (Internal error: Error finding id)
|
|
description: 2026-05-03 diagnostic state. Three days of CPU/MPS time on the bge-m3 chamber-library mine produced 933,576 drawers in storage but functional semantic recall is broken. Real loss for the steward.
|
|
type: project
|
|
originSessionId: 8d9ac240-47df-4578-aec1-a9dd32965027
|
|
---
|
|
## Diagnostic state (2026-05-03 evening)
|
|
|
|
**The good (storage)**
|
|
- `~/.mempalace/palace/chroma.sqlite3` is 6.3 GB
|
|
- `mempalace_status` returns instantly: `total_drawers: 933,576`
|
|
- `mempalace_list_wings` returns instantly: `chamber_library: 933,576`
|
|
- All embeddings are dimension 1024 (bge-m3 confirmed)
|
|
- Two Chroma collections exist: `mempalace_drawers` (1024d), `mempalace_closets` (1024d)
|
|
- Last write to chroma.sqlite3: today (2026-05-03 17:34)
|
|
- Mining process is NOT running — terminated cleanly per machine cold
|
|
|
|
**The bad (search)**
|
|
- `mempalace_search` with any wing-scoped query returns immediately with:
|
|
```
|
|
Search error: Error executing plan: Internal error: Error finding id
|
|
```
|
|
- Tried two queries — both errored:
|
|
- "Athenaeus herons curved necks seed-gathering grouse warblers sukallides fig season" (steward's verbatim test passage)
|
|
- "Bringhurst" (single keyword)
|
|
- Both returned the SAME error string → not a query-content issue
|
|
|
|
**Working hypothesis**
|
|
The bge-m3 + MPS mine produced the storage layer (Chroma's sqlite + embedding tables) but the LanceDB / vector index that backs `search` is incomplete or corrupted. Possible causes:
|
|
- Mine completed embeddings but never finished building/persisting the search index
|
|
- Index segment file got truncated or had write failure mid-build
|
|
- bge-m3 dimension change (was 384 with mxbai, now 1024) requires a fresh index that wasn't fully built
|
|
|
|
## What this cost
|
|
|
|
**Three days of CPU/MPS compute**, per steward's accounting. The steward wrapped the prior session having already deferred the mine for fresh-session restart on MPS to get 5-10x speedup. That work appears to have run to completion on the storage side (drawer count went from 501k @ pause-point on 2026-05-01 to 934k now) — but the recall function the mine was meant to enable is non-functional.
|
|
|
|
This is **not** a "non-fatal" failure. The whole point of the mine was searchable verbatim recall via MCP. Storage without search is useless storage. The contamination-pattern frame: a system that reports "ready" via its own counters (status returns 933k drawers, list_wings returns the wing) but cannot deliver the actual function it was built for.
|
|
|
|
## Important honesty correction
|
|
|
|
When the steward said "40 minutes for mempalace queries — must be slow," that was incorrect framing on my part. Mempalace queries themselves return in milliseconds — `status` and `list_wings` are instant; `search` returns the error string immediately. The 40 minutes was me getting deflected by a cascade of UI regressions (audit-induced) and never coming back to characterize the search failure. **Mempalace is not slow; it is broken.** That distinction matters because the diagnosis differs (slow → optimization; broken → repair).
|
|
|
|
## Possible paths forward (not yet tried)
|
|
|
|
1. **Diagnose first, don't re-mine.** Inspect the Chroma `segments` table + on-disk index files (`palace/<segment-uuid>/`) to see whether the index files are present, what state they're in, and whether the search error refers to a specific missing segment ID.
|
|
2. **Try `--force-rebuild-index`** or equivalent mempalace mine flag (if one exists) — might rebuild the search index without re-embedding. Saves the 3 days.
|
|
3. **Mine to a fresh palace path**, keep the broken one for forensics. Worst case scenario; lose the 3 days.
|
|
4. **File issue upstream** — Seb / mempalace repo. The "Internal error: Error finding id" is opaque enough that it warrants the maintainer's eyes.
|
|
|
|
## Steward signal
|
|
|
|
> "Please take note of the mempalace debugging. Three days of time that mining cost me."
|
|
> "40 minutes. I smell something rotten with mempalace. When you call it it takes forever."
|
|
|
|
The frustration is appropriate. Don't minimize. Don't reach for "still useful" framings. The mine cost real time and the result is not what was promised.
|