8.1 KiB
name, description, metadata
| name | description | metadata | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| source-library-link-to-chamber-dedupe-steward-goal | The steward's master source library is ~/Documents/___The Library [ePub_AWZ3]/ (2190 ebooks: 1573 epub, 267 azw3, 335 pdf, 15 mobi; 180k total files; messy nested subfolders). Eventual goal: LINK it to the chamber (canonical_texts ↔ source provenance) and DEDUPE the messy folder. The fuzzy source-matcher built 2026-06-29 is the seed of both. |
|
Location: /Users/davidglidden/Documents/___The Library [ePub_AWZ3]/ — the steward's master ebook library. 2190 ebooks (1573 .epub, 267 .azw3, 335 .pdf, 15 .mobi), 180,567 total files, deeply/messily nested. Notable subfolders the steward pre-organized: _2026 chamber source cleanup/ (categorized 1. PDF, 2. EPUB/<tradition>), tmp calibre library for the chamber/ (clean Calibre Author/Title (id)/Title - Author.ext structure — the most matchable), The Library [ePub_AWZ3]/ (nested dup of the root, also Calibre-ish), __PDF/, superpower mine/, New texts may 2026/. The steward asserts the originals for ALL chamber sources are in here somewhere.
Steward goal (stated 2026-06-29): (1) LINK the two libraries — chamber canonical_texts/ ↔ their source files (a provenance index: every canonical work → the exact source it was converted from). (2) DEDUPE the messy folder (z-library dupes, Calibre dupes, format dupes epub+azw3+pdf of the same work, the root↔The Library [ePub_AWZ3]/ nesting dup).
Seed already built (2026-06-29): a fuzzy source-matcher (token-overlap of title/author keywords vs library filename stems) — used to match the 34 reconvert-flagged works → sources (24 found, 10 needed manual locate). Output: chamber-library/_curation/reconvert-queue-2026-06-29.md. This same matcher generalizes to BOTH goals: run it over ALL ~1290 canonical works → the link index; run it within the library (filename clustering) → duplicate clusters for dedupe.
How this connects to the reconversion work: the immediate driver was that ~34 of the ~69 remaining cruft files are the reconvert-flagged set (_curation/reconvert-list-2026-06-16.txt) — already judged "reconvert, don't patch." The link index makes reconversion executable (each flagged work → its source). See feedback-character-as-image-hazard (pre-check sources before converting), the conversion runbook, and the session ledger 2026-06-29 for the backlog map.
CANONICAL PROVENANCE ARCHIVE ESTABLISHED (2026-07-02): the destination for confirmed source originals is now decided — iCloud Drive / Chamber Sources/ (~/Library/Mobile Documents/com~apple~CloudDocs/Chamber Sources/). Steward-authorized. OUTSIDE the repo (text-only), iCloud = local+cloud, 2TB plan (storage no constraint). Layout = flat + MANIFEST.md (the archive's self-describing index: filename→sha256→graduated chamber text→edition notes). Filenames key to the graduated slug. ⚠ Must be pinned "Keep Downloaded" in Finder — iCloud "Optimize Storage" silently evicts to dataless stubs otherwise (found a live dataless file on Desktop; that's why NOT the Desktop surface). First intake: Jonas (2 scans: IA-color-360dpi = provenance-of-record; bitonal-600dpi = hi-res verify backup), Vico epub (Marsh/Grafton), Heidegger pdf (Hofstadter/Harper). The in-repo per-text sidecar (spec §VI source_sha256) stays DEFERRED per PENDING-42 §VI; MANIFEST.md is the interim index; reconcile with _curation/chamber-source-link.md later. The 25GB master library folds in here per the link+dedupe pass. Heidegger repair candidate: graduated .md title garbled "Poetry, 'bonbuoD Thought" (="Language") — clean source now archived to verify/repair.
Caution (matcher limits): fuzzy filename matching misses French/diacritic titles & differently-named editions (Mythologiques, Les Fleurs du Mal, Peirce, Neumann all returned no match despite likely being present). A real link index should also read EPUB/PDF internal metadata (dc:title/dc:creator), not just filenames. And verify edition/translation identity (Foucault: the library's "Madness and Civilization" is the ABRIDGED EN, ≠ the canonical FR "Histoire de la folie") — wrong-edition is worse than no-match.
THE FULL SEQUENCE (steward-restated 2026-07-24 — the ordering is load-bearing): (1) Chamber Sources must hold the source original of EVERY canon document — the invariant feedback-resolve-source-through-chamber-sources. (2) ONLY once we are SURE Chamber Sources holds all originals do we dedupe the messy master library against it. (3) Whatever files REMAIN in the messy folder after dedupe are evaluated for eventual admission to the Chamber Library. So the source-completeness invariant is the precondition for the dedupe, which is the precondition for the admission triage. Do them in that order.
PHASE ALIGNMENT + DECISIONS (2026-07-24) — maps onto library-consolidation-plan-2026-06-30.md:
- Phase 2 "build Chamber Originals" = DONE today (Chamber Sources holds every non-Loeb original + Loeb compound; the source-completeness work above).
- Phase 1 dedupe: live library = 0 byte-identical dupes (July-1 pass was complete; re-ran
index_library.py→propose_library_dedupe.pyfiltered to live/non-_trash= 0 Tier-A; Tier B 20 review · Tier C 206 flag-only kept). The July-1 quarantine_trash/dedupe-20260701/= 115 files / 935 MB, op-record verified byte-identical, each keeper live → steward emptying it (mv … ~/.Trash/) = the "remove clear byte-dupes" step. - Phase 3 "build the Library" = the SILVA consolidation (NEXT concentrated bite, NOT rushed). Target folder
~/Documents/Silva/(steward-chosen name — Latin silva, a woodland of sources to draw timber from; most never canon; deliberately NOT "Chamber"/"canon"). Books only — classify-first:.epub/.azw3/.mobi= books wholesale; PDFs triaged by page-count + size + junk-name patterns (demonstrated on the 528 Desktop+Downloads scatter → 136 book / 320 non-book [tax forms, birth certs, IMSLP scores — LEAVE in place] / 72 eyeball; refine with music-score + work-doc patterns — Beethoven-manuscript/AldineXXI-spec slipped into BOOK by page-count). Chamber Sources stays separate as the immutable archive. Method: build a Phase-3 dry-run move-map (author-organized, flatten the nestedThe Library [ePub_AWZ3]/The Library [ePub_AWZ3]/, collision+dedup handling, count-reconciliation) → steward cuts → reversible moves. RESOLVED design decision (steward 2026-07-24): SILVA = EVERYTHING — the complete reservoir of ALL source books, no subtraction. Chamber Sources = the explicit, immutable duplicate copies of (only) the chamber canon's sources (the ~348 overlap is INTENTIONAL — a deliberate sha-locked archive duplicate of what became canon, separate from the browse-reservoir). So Silva never reasons about "is this already archived"; it holds it all, and Chamber Sources stays the untouched provenance archive. (Supersedes the plan's original "Library = remainder MINUS Chamber Originals".) No consolidation tool exists yet (scripts/ has none) — build it at the Silva bite.
CENSUS 2026-07-24 (read-only, resolve_archived_source over all canonical_texts/**/*.md): 1,297 canonicals · 344 pinned (source entry in Chamber Sources, all materialized) · 940 loeb-* (sourced from the compound loeb-dsl-537-volumes.dsl artifact already in Chamber Sources — the ORIGINAL is held; per-work attach entries are created as each graduates through the one-door, so NOT a missing-original gap) · 13 non-Loeb GENUINELY unsourced = the real gap to close for the invariant: mla-9th-ed, unesco-2nd-ed, warde-crystal-goblet, la-citadelle-interieure-hadot, the-book-before-printing-diringer, technics-and-human-development-mumford, the-pentagon-of-power-mumford, la-philosophie-comme-maniere-de-vivre-hadot, la-poe-tique-de-la-re-verie, the-red-book, moby-dick, the-dispossessed-leguin, airs. (Was 14 — the Levi omnibus mis-pin was fixed this session: re-pinned to the Complete Works EPUB, sha 1be71a50.)