--- name: EPUB → chamber-library conversion pipeline (with structural repair) description: For scholarly EPUBs where pandoc alone loses chapter structure, repair the EPUB source first via TOC-driven heading promotion, then convert with pandoc + strip_cruft. Built 2026-05-13. type: project originSessionId: a5411fd1-4a96-43e7-aa1a-1a50d8951a8f --- For converting EPUBs from the messy folder (`~/Documents/___The Library [ePub_AWZ3]/`) into chamber-library, the durable pipeline is: ``` EPUB → repair_epub_headings.py → pandoc -t gfm-raw_html → strip_cruft.py → chamber-library (TOC-driven heading promotion) (clean MD) (final residue) ``` Each stage: 1. **`~/_Dev/chamber-library/scripts/repair_epub_headings.py`** — uses the EPUB's own NCX/NAV (machine-readable TOC) as authoritative source for chapter structure. Promotes styled `

` chapter titles to proper `` tags. Publisher-convention-independent. Output is a new EPUB at a specified path; original is never modified. Usage: `repair_epub_headings.py INPUT.epub OUTPUT.epub [--max-depth N] [--verbose]` Uses dedicated venv at `~/.local/chamber-tools-venv/` (bs4 + lxml). 2. **`pandoc -t gfm-raw_html --wrap=preserve --extract-media=DIR -o OUT.md REPAIRED.epub`** — convert to clean Markdown. The `-raw_html` suffix suppresses HTML scaffolding preservation (the lesson from the Polastrons/Obrist work earlier same day). 3. **`~/_Dev/chamber-library/scripts/strip_cruft.py OUT.md --apply --samples 0`** — final cleanup pass for any pandoc class residue + link/image attribute stripping. Writes backup to `~/_Dev/chamber-library/_backups/YYYY-MM-DD/`. **Tested 2026-05-13 on three different publisher conventions, all successful:** - Rowman & Littlefield (Bachelard *Dialectic of Duration*): `

/

` encoding, anchor IDs in NCX. 19/20 promoted; final MD has 19 H1 headings including all 8 chapter titles ("Relaxation and Nothingness", etc.). - Routledge (Weil *Gravity and Grace*): chapter-per-file encoding, single styled-title paragraph per file. 39/46 promoted (front matter skipped); final MD has 38 chapter H1 headings. - Bloomsbury (Gadamer *Truth and Method*): `

/

` for chapters + `

/

` for parts, no fragments in NCX. 33/35 promoted; final MD has 10 H1 + 16 H2 = 26 headings (3 Parts + 5 main chapters + 6 Appendices + 2 Supplements + Translator's Preface). **Why the tool is publisher-convention-independent:** - Uses NCX/NAV (required by EPUB spec) as authoritative TOC source - Matches by anchor id (preferred) OR by text content (fallback) - Text matching handles leading numeration in many forms: arabic ("1 Title"), Roman ("III: Title"), English words ("Part One Title") - Number-stub fall-through: when anchor target is just a chapter number (e.g., "Chapter 1"), falls through to text matching to find the substantive title - Inner `` wrapper stripping by default so pandoc produces clean `# Title` not `# [Title](anchor)` **Important context — the failure mode this tool fixes:** Many scholarly EPUBs (especially from typesetting workflows like Adobe InDesign export) encode chapter titles as styled paragraphs: ```html

Chapter 1

Relaxation and Nothingness

``` rather than: ```html

Chapter 1: Relaxation and Nothingness

``` When pandoc converts these with default flags, it sees only paragraphs — no headings to promote. Output is structurally flat regardless of the conversion flag. The chamber-library audit flagged 127 files in this state (RECONVERT bucket). This tool fixes the source EPUB first so any subsequent conversion (now or later) inherits proper structure. **Filing discipline (steward's directive 2026-05-13):** - **EPUBs and MD conversions live in DIFFERENT locations** — never mix - **Repaired EPUBs**: `~/Documents/__Chamber Source EPUBs/repaired-YYYY-MM-DD/.epub` - File apart from chamber-library entirely; this is source-material territory, not converted-text territory - Naming matches the chamber MD filename (e.g., `the_dialectic_of_duration_bachelard.epub` for chamber `the_dialectic_of_duration_bachelard.md`) - Double-underscore prefix on the parent folder follows steward's convention for important folders like `__must read/` - **MD conversions**: filed at the canonical chamber-library path (e.g., `~/_Dev/chamber-library/converted_texts/traditions/critical_modernity/phenomenology/the_dialectic_of_duration_bachelard.md`) - **Original EPUBs** in `~/Documents/___The Library [ePub_AWZ3]/` and `~/Desktop/New texts may 2026/` are never modified - **Existing chamber file backups** (when replacing a broken existing MD): `~/_Dev/chamber-library/_backups/YYYY-MM-DD/.preconversion-backup` - `mempalace mine` automatically detects mtime change on the chamber MD → deletes old drawers → re-mines clean content on its next iteration through that directory **For the broader RECONVERT queue (127 files):** This tool is the right shape for any EPUB-source RECONVERT file. For PDF-source files, docling is the conversion tool (per the toolset study 2026-05-13). For files where the source can't be found in `~/Documents/___The Library [ePub_AWZ3]/` or `~/Desktop/New texts may 2026/`, the file stays in the chamber as-is until source surfaces.