31 lines
8.1 KiB
Markdown
31 lines
8.1 KiB
Markdown
---
|
|
name: source-library-link-to-chamber-dedupe-steward-goal
|
|
description: "The steward's master source library is ~/Documents/___The Library [ePub_AWZ3]/ (2190 ebooks: 1573 epub, 267 azw3, 335 pdf, 15 mobi; 180k total files; messy nested subfolders). Eventual goal: LINK it to the chamber (canonical_texts ↔ source provenance) and DEDUPE the messy folder. The fuzzy source-matcher built 2026-06-29 is the seed of both."
|
|
metadata:
|
|
node_type: memory
|
|
type: project
|
|
originSessionId: 604b3d1d-6111-48f2-9683-2dfba02d5b6b
|
|
modified: 2026-07-24T08:44:06.397Z
|
|
---
|
|
|
|
**Location:** `/Users/davidglidden/Documents/___The Library [ePub_AWZ3]/` — the steward's master ebook library. 2190 ebooks (1573 .epub, 267 .azw3, 335 .pdf, 15 .mobi), 180,567 total files, deeply/messily nested. Notable subfolders the steward pre-organized: `_2026 chamber source cleanup/` (categorized `1. PDF`, `2. EPUB/<tradition>`), `tmp calibre library for the chamber/` (clean Calibre `Author/Title (id)/Title - Author.ext` structure — the most matchable), `The Library [ePub_AWZ3]/` (nested dup of the root, also Calibre-ish), `__PDF/`, `superpower mine/`, `New texts may 2026/`. The steward asserts the originals for ALL chamber sources are in here somewhere.
|
|
|
|
**Steward goal (stated 2026-06-29):** (1) LINK the two libraries — chamber `canonical_texts/` ↔ their source files (a provenance index: every canonical work → the exact source it was converted from). (2) DEDUPE the messy folder (z-library dupes, Calibre dupes, format dupes epub+azw3+pdf of the same work, the root↔`The Library [ePub_AWZ3]/` nesting dup).
|
|
|
|
**Seed already built (2026-06-29):** a fuzzy source-matcher (token-overlap of title/author keywords vs library filename stems) — used to match the 34 reconvert-flagged works → sources (24 found, 10 needed manual locate). Output: `chamber-library/_curation/reconvert-queue-2026-06-29.md`. This same matcher generalizes to BOTH goals: run it over ALL ~1290 canonical works → the link index; run it within the library (filename clustering) → duplicate clusters for dedupe.
|
|
|
|
**How this connects to the reconversion work:** the immediate driver was that ~34 of the ~69 remaining cruft files are the reconvert-flagged set (`_curation/reconvert-list-2026-06-16.txt`) — already judged "reconvert, don't patch." The link index makes reconversion executable (each flagged work → its source). See [[feedback-character-as-image-hazard]] (pre-check sources before converting), the conversion runbook, and the session ledger 2026-06-29 for the backlog map.
|
|
|
|
**CANONICAL PROVENANCE ARCHIVE ESTABLISHED (2026-07-02):** the *destination* for confirmed source originals is now decided — **`iCloud Drive / Chamber Sources/`** (`~/Library/Mobile Documents/com~apple~CloudDocs/Chamber Sources/`). Steward-authorized. OUTSIDE the repo (text-only), iCloud = local+cloud, 2TB plan (storage no constraint). Layout = **flat + `MANIFEST.md`** (the archive's self-describing index: filename→sha256→graduated chamber text→edition notes). Filenames key to the graduated slug. ⚠ **Must be pinned "Keep Downloaded" in Finder** — iCloud "Optimize Storage" silently evicts to dataless stubs otherwise (found a live dataless file on Desktop; that's why NOT the Desktop surface). First intake: Jonas (2 scans: IA-color-360dpi = provenance-of-record; bitonal-600dpi = hi-res verify backup), Vico epub (Marsh/Grafton), Heidegger pdf (Hofstadter/Harper). The in-repo per-text sidecar (spec §VI `source_sha256`) stays DEFERRED per PENDING-42 §VI; MANIFEST.md is the interim index; reconcile with `_curation/chamber-source-link.md` later. The 25GB master library folds in here per the link+dedupe pass. **Heidegger repair candidate:** graduated .md title garbled "Poetry, 'bonbuoD Thought" (="Language") — clean source now archived to verify/repair.
|
|
|
|
**Caution (matcher limits):** fuzzy filename matching misses French/diacritic titles & differently-named editions (Mythologiques, Les Fleurs du Mal, Peirce, Neumann all returned no match despite likely being present). A real link index should also read EPUB/PDF *internal metadata* (dc:title/dc:creator), not just filenames. And verify edition/translation identity (Foucault: the library's "Madness and Civilization" is the ABRIDGED EN, ≠ the canonical FR "Histoire de la folie") — wrong-edition is worse than no-match.
|
|
|
|
**THE FULL SEQUENCE (steward-restated 2026-07-24 — the ordering is load-bearing):** (1) **Chamber Sources must hold the source original of EVERY canon document** — the invariant [[feedback-resolve-source-through-chamber-sources]]. (2) **ONLY once we are SURE Chamber Sources holds all originals** do we dedupe the messy master library against it. (3) Whatever files REMAIN in the messy folder after dedupe are **evaluated for eventual admission to the Chamber Library**. So the source-completeness invariant is the *precondition* for the dedupe, which is the *precondition* for the admission triage. Do them in that order.
|
|
|
|
**PHASE ALIGNMENT + DECISIONS (2026-07-24) — maps onto `library-consolidation-plan-2026-06-30.md`:**
|
|
- **Phase 2 "build Chamber Originals" = DONE today** (Chamber Sources holds every non-Loeb original + Loeb compound; the source-completeness work above).
|
|
- **Phase 1 dedupe: live library = 0 byte-identical dupes** (July-1 pass was complete; re-ran `index_library.py`→`propose_library_dedupe.py` filtered to live/non-`_trash` = 0 Tier-A; Tier B 20 review · Tier C 206 flag-only kept). The July-1 quarantine `_trash/dedupe-20260701/` = **115 files / 935 MB**, op-record verified byte-identical, each keeper live → **steward emptying it** (`mv … ~/.Trash/`) = the "remove clear byte-dupes" step.
|
|
- **Phase 3 "build the Library" = the SILVA consolidation (NEXT concentrated bite, NOT rushed).** Target folder **`~/Documents/Silva/`** (steward-chosen name — Latin *silva*, a woodland of sources to draw timber from; most never canon; deliberately NOT "Chamber"/"canon"). **Books only** — classify-first: `.epub/.azw3/.mobi` = books wholesale; PDFs triaged by page-count + size + junk-name patterns (demonstrated on the 528 Desktop+Downloads scatter → 136 book / 320 non-book [tax forms, birth certs, IMSLP scores — LEAVE in place] / 72 eyeball; refine with music-score + work-doc patterns — Beethoven-manuscript/AldineXXI-spec slipped into BOOK by page-count). Chamber Sources stays separate as the immutable archive. Method: build a Phase-3 dry-run move-map (author-organized, flatten the nested `The Library [ePub_AWZ3]/The Library [ePub_AWZ3]/`, collision+dedup handling, count-reconciliation) → steward cuts → reversible moves. **RESOLVED design decision (steward 2026-07-24): SILVA = EVERYTHING** — the complete reservoir of ALL source books, no subtraction. **Chamber Sources = the explicit, immutable duplicate copies of (only) the chamber canon's sources** (the ~348 overlap is INTENTIONAL — a deliberate sha-locked archive duplicate of what became canon, separate from the browse-reservoir). So Silva never reasons about "is this already archived"; it holds it all, and Chamber Sources stays the untouched provenance archive. (Supersedes the plan's original "Library = remainder MINUS Chamber Originals".) No consolidation tool exists yet (scripts/ has none) — build it at the Silva bite.
|
|
|
|
**CENSUS 2026-07-24 (read-only, `resolve_archived_source` over all `canonical_texts/**/*.md`):** 1,297 canonicals · **344 pinned** (source entry in Chamber Sources, all materialized) · **940 loeb-*** (sourced from the compound `loeb-dsl-537-volumes.dsl` artifact already in Chamber Sources — the ORIGINAL is held; per-work attach entries are created as each graduates through the one-door, so NOT a missing-original gap) · **13 non-Loeb GENUINELY unsourced** = the real gap to close for the invariant: `mla-9th-ed`, `unesco-2nd-ed`, `warde-crystal-goblet`, `la-citadelle-interieure-hadot`, `the-book-before-printing-diringer`, `technics-and-human-development-mumford`, `the-pentagon-of-power-mumford`, `la-philosophie-comme-maniere-de-vivre-hadot`, `la-poe-tique-de-la-re-verie`, `the-red-book`, `moby-dick`, `the-dispossessed-leguin`, `airs`. (Was 14 — the Levi omnibus mis-pin was fixed this session: re-pinned to the Complete Works EPUB, sha `1be71a50`.)
|