🔧 Auto-commit from sysupdate on 2026-05-19 21:49
This commit is contained in:
David F Glidden
@@ -0,0 +1,208 @@
|
||||
---
|
||||
name: Pending work for 2026-05-06 — substrate finishing tasks
|
||||
description: Concrete to-dos carrying over from 2026-05-05's substrate restoration day. Order by dependency. All small (under an hour each). Most blocked behind a Claude Code restart that brings MCP back.
|
||||
type: project
|
||||
originSessionId: a156edbb-0c3d-4285-b3ed-f5a45b8dda66
|
||||
---
|
||||
# Pending work for 2026-05-06
|
||||
|
||||
The 2026-05-05 substrate restoration left a clean working state but ~5 small tasks pulling forward. Most are blocked behind a Claude Code restart (which respawns MCP). Order matters — earlier tasks unblock later ones.
|
||||
|
||||
---
|
||||
|
||||
## 1. Wake-up checks (before anything substantive)
|
||||
|
||||
Before touching anything, check current state:
|
||||
|
||||
```bash
|
||||
# Mining: did it complete overnight?
|
||||
sqlite3 ~/.mempalace/palace/chroma.sqlite3 \
|
||||
'SELECT COUNT(*) FROM embeddings e JOIN segments s ON e.segment_id=s.id JOIN collections c ON s.collection=c.id WHERE c.name="mempalace_drawers";'
|
||||
# Expect: ~600k+ if mine completed; ~480-560k if still in Loeb tail
|
||||
tail -10 /tmp/mine_2026-05-05.log
|
||||
ps -ef | grep "mempalace mine" | grep -v grep # empty if done
|
||||
|
||||
# Marker: status of 15-PDF batch (4 Alexander Nature of Order books are the tail)
|
||||
tail -5 /tmp/marker_2026-05-05.log
|
||||
ls ~/_Dev/chamber-library/converted_texts/_pending_chamber_metadata/ # number of completed books
|
||||
ps -ef | grep marker_single | grep -v grep # empty if done
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Restart Claude Code (unblocks MCP, hook end-to-end, diary_write)
|
||||
|
||||
The MCP server (`mempalace-mcp`) was killed during this morning's max_seq_id repair and never restarted. It auto-respawns when Claude Code starts. The hook patch is verified working but the *save* it forces routes through MCP — so until Claude Code restarts, no diary auto-saves can complete.
|
||||
|
||||
**A clean Claude Code restart restores three things at once**: MCP server up, fresh session for hook to count exchanges from, and the new fresh-palace substrate fully connected.
|
||||
|
||||
Recommended timing: after wake-up checks confirm mining/Marker state, restart Claude Code. Either:
|
||||
- Quit and re-launch the app
|
||||
- Or (in a fresh terminal): `pkill -f mempalace-mcp` then start a new Claude Code session
|
||||
|
||||
---
|
||||
|
||||
## 3. Verify hook patch on first post-restart turn
|
||||
|
||||
After the first assistant turn in the new session, the Stop hook should fire with a real session_id and exchange count:
|
||||
|
||||
```bash
|
||||
tail -3 ~/.mempalace/hook_state/hook.log
|
||||
```
|
||||
|
||||
Expected: `[HH:MM:SS] Session <UUID>: <N> exchanges, <N> since last save` — NOT `Session unknown: 0 exchanges`.
|
||||
|
||||
If still showing "Session unknown" → patch didn't survive a `git pull` (mempalace got updated overnight). Re-apply: see §7 below.
|
||||
|
||||
If new format → fix is verified end-to-end.
|
||||
|
||||
---
|
||||
|
||||
## 4. Supplementary mine of new shelves added 2026-05-05 (NOT picked up by yesterday's mine)
|
||||
|
||||
The 2026-05-05 mine had hardcoded TARGETS. Content added during the day that the mine won't have picked up:
|
||||
|
||||
**Shelves NOT in the original TARGETS array (need full mine):**
|
||||
- `traditions/contemporary_voices/letters_and_memory/` (6 files: Yates *Art of Memory*, Manguel ×3, Adler *How to Read a Book*, Foer *Moonwalking*)
|
||||
- `traditions/music_performance/` (Taruskin OHWM 5-vol, Heartz, Swafford ×2 — joining existing 3 files; need full re-mine of shelf since the existing 3 were never mined)
|
||||
- `traditions/history/` (Davies *Europe*)
|
||||
|
||||
**Shelves where mining phase already completed before files were added (need re-mine):**
|
||||
- `collections/style_guides/` — Strunk added after phase B done
|
||||
- `traditions/art_culture/visual_culture/` — 3 Solnit titles added after phase done
|
||||
|
||||
**Files added late to existing TARGETS (may or may not have been picked up depending on mine timing):**
|
||||
- `traditions/literature/classical/` — Rilke ×3, Stevens, Whitman, Eliot ×2, Thoreau (added before the literature/classical phase, should ride along)
|
||||
|
||||
Quick supplementary mine command (after Claude Code restart so MCP is up — though `mempalace mine` works via CLI regardless):
|
||||
|
||||
```bash
|
||||
# All into wing chamber_library to match existing curation
|
||||
for target in \
|
||||
collections/style_guides \
|
||||
traditions/contemporary_voices/letters_and_memory \
|
||||
traditions/music_performance \
|
||||
traditions/art_culture/visual_culture \
|
||||
traditions/history; do
|
||||
mempalace mine "$HOME/_Dev/chamber-library/converted_texts/$target" --wing chamber_library
|
||||
done
|
||||
```
|
||||
|
||||
(`style_guides` could use a different wing if the existing 8 are already mined under a different wing — check first.)
|
||||
|
||||
Expected runtime: ~5-10 min total. mempalace's dedup will skip files already mined.
|
||||
|
||||
---
|
||||
|
||||
## 5. OCR pipeline for two scan-only epubs
|
||||
|
||||
Two epubs from yesterday's ARC slate were image-only scans (no extractable text). They need OCR.
|
||||
|
||||
**Files**:
|
||||
- `Ross W Duffin/How Equal Temperament Ruined Harmony (and Whys You SHould Care) (737)/...epub` (33 MB)
|
||||
- `Treatise On The Fundamentals Of Violin Playing/Leopold Mozart (664)/...epub` (64 MB) — Leopold Mozart's treatise
|
||||
|
||||
**Pipeline** (Calibre's ebook-convert → PDF → Marker → mine):
|
||||
|
||||
```bash
|
||||
# Setup: symlink if not done already
|
||||
ln -s /Applications/calibre.app/Contents/MacOS/ebook-convert /opt/homebrew/bin/ebook-convert
|
||||
|
||||
LIB="/Users/davidglidden/Documents/The Library [ePub_AWZ3]/The Library [ePub_AWZ3]"
|
||||
TMP=/tmp/scan-pdfs-2026-05-06
|
||||
mkdir -p "$TMP"
|
||||
|
||||
ebook-convert "$LIB/Ross W Duffin/.../...epub" "$TMP/duffin-equal-temperament.pdf"
|
||||
ebook-convert "$LIB/Treatise On The Fundamentals Of Violin Playing/.../...epub" "$TMP/leopold-mozart-violin-treatise.pdf"
|
||||
|
||||
# Then Marker (after current 15-PDF batch wraps):
|
||||
~/.local/marker-venv/bin/marker_single "$TMP/duffin-equal-temperament.pdf" \
|
||||
--output_dir "$HOME/_Dev/chamber-library/converted_texts/traditions/music_performance"
|
||||
~/.local/marker-venv/bin/marker_single "$TMP/leopold-mozart-violin-treatise.pdf" \
|
||||
--output_dir "$HOME/_Dev/chamber-library/converted_texts/traditions/music_performance"
|
||||
|
||||
# Then mine (or include in §4's supplementary mine if running after this):
|
||||
mempalace mine "$HOME/_Dev/chamber-library/converted_texts/traditions/music_performance" --wing chamber_library
|
||||
```
|
||||
|
||||
Estimated time: ~30 sec each for ebook-convert; ~1-2 hr each for Marker depending on page count + dpi. Schedule after main Marker batch finishes.
|
||||
|
||||
---
|
||||
|
||||
## 6. Mine the Marker-converted superpower PDFs
|
||||
|
||||
The 15-PDF Marker batch from 2026-05-05 places output at:
|
||||
```
|
||||
~/_Dev/chamber-library/converted_texts/_pending_chamber_metadata/<book>/
|
||||
```
|
||||
|
||||
When Marker finishes (likely sometime tomorrow, 4 Alexander Nature of Order books are the slow tail), mine these into wing chamber_library:
|
||||
|
||||
```bash
|
||||
mempalace mine ~/_Dev/chamber-library/converted_texts/_pending_chamber_metadata --wing chamber_library
|
||||
```
|
||||
|
||||
Estimated: 30-60 min depending on output sizes.
|
||||
|
||||
The 15 books being added: Hochuli, Müller-Brockmann, Smeijers, Morison, Rutter, Bringhurst *Solid Form*, Alexander *Timeless Way* + *Oregon Experiment* + 4 Nature of Order books, Bachelard ×3 (English versions of Fire/Air/Reverie).
|
||||
|
||||
After mining, consider **moving the converted markdowns out of `_pending_chamber_metadata/` into proper chamber-library subdirs** (e.g., Hochuli → `collections/typography/`, Bachelard English → `traditions/critical_modernity/phenomenology/`, Alexander → `traditions/contemporary_voices/environmental/`). This is post-mining cleanup; staging dir is meant to be temporary per yesterday's design.
|
||||
|
||||
---
|
||||
|
||||
## 7. Re-apply hook patch if mempalace was pulled overnight
|
||||
|
||||
Local patch in `~/_Dev/mempalace/hooks/mempal_save_hook.sh` will be reverted by any `git pull`. To re-apply:
|
||||
|
||||
Around line 85, replace:
|
||||
```bash
|
||||
mapfile -t _mempal_parsed < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
|
||||
[python script]
|
||||
" 2>/dev/null)
|
||||
```
|
||||
|
||||
with:
|
||||
```bash
|
||||
_mempal_parsed=()
|
||||
while IFS= read -r _line; do
|
||||
_mempal_parsed+=("$_line")
|
||||
done < <(echo "$INPUT" | "$MEMPAL_PYTHON_BIN" -c "
|
||||
[same python script]
|
||||
" 2>/dev/null)
|
||||
```
|
||||
|
||||
The detailed reasoning + the exact diff lives in the upstream issue draft at `~/_Dev/mempalace/.local-data/upstream-issue-bash32-mapfile-2026-05-05.md`.
|
||||
|
||||
---
|
||||
|
||||
## 8. File upstream issues (drafts ready)
|
||||
|
||||
Two drafts in `~/_Dev/mempalace/.local-data/`:
|
||||
|
||||
- `upstream-issue-draft-2026-05-05.md` — `mempalace mine` reports drawer counts at submission, not commit (lying-counter; silent-drop on max_seq_id poisoning)
|
||||
- `upstream-issue-bash32-mapfile-2026-05-05.md` — Stop hook silently fails on macOS bash 3.2 (`mapfile` incompatibility)
|
||||
|
||||
Both reviewed and ready. File via `gh issue create` against the appropriate mempalace repo. Steward to authorize submission.
|
||||
|
||||
---
|
||||
|
||||
## 9. gemma4:e4b ollama model decision (carried from 2026-05-04)
|
||||
|
||||
9.6 GB unused model. Yesterday's open horizon. Triage: kept or remove? Quick `ollama show gemma4:e4b` + decision.
|
||||
|
||||
---
|
||||
|
||||
## Order of attack (recommended)
|
||||
|
||||
1. Wake-up checks (§1) — 2 min
|
||||
2. Restart Claude Code (§2) — 1 min
|
||||
3. Verify hook (§3) — 30 sec
|
||||
4. Re-apply hook patch if needed (§7) — 2 min if needed
|
||||
5. Start ebook-convert (§5) for the two scan epubs while doing other work — 1 min to kick off
|
||||
6. Supplementary mine (§4) — 5-10 min
|
||||
7. When Marker batch wraps: mine converted PDFs (§6) — 30-60 min
|
||||
8. When ebook-converts finish: queue Marker on the 2 scans (§5) — runs in background
|
||||
9. File upstream issues (§8) — 10 min
|
||||
10. gemma4:e4b decision (§9) — 5 min
|
||||
|
||||
Then: actually working on ARC again with the substrate fully restored, the panel deeper than ever, and the hook auto-save chain operative end-to-end.
|
||||
Reference in New Issue
Block a user