69 lines
5.3 KiB
Markdown
69 lines
5.3 KiB
Markdown
---
|
|
name: EPUB → chamber-library conversion pipeline (with structural repair)
|
|
description: For scholarly EPUBs where pandoc alone loses chapter structure, repair the EPUB source first via TOC-driven heading promotion, then convert with pandoc + strip_cruft. Built 2026-05-13.
|
|
type: project
|
|
originSessionId: a5411fd1-4a96-43e7-aa1a-1a50d8951a8f
|
|
---
|
|
For converting EPUBs from the messy folder (`~/Documents/___The Library [ePub_AWZ3]/`) into chamber-library, the durable pipeline is:
|
|
|
|
```
|
|
EPUB → repair_epub_headings.py → pandoc -t gfm-raw_html → strip_cruft.py → chamber-library
|
|
(TOC-driven heading promotion) (clean MD) (final residue)
|
|
```
|
|
|
|
Each stage:
|
|
|
|
1. **`~/_Dev/chamber-library/scripts/repair_epub_headings.py`** — uses the EPUB's own NCX/NAV (machine-readable TOC) as authoritative source for chapter structure. Promotes styled `<p class="…">` chapter titles to proper `<h*>` tags. Publisher-convention-independent. Output is a new EPUB at a specified path; original is never modified.
|
|
|
|
Usage: `repair_epub_headings.py INPUT.epub OUTPUT.epub [--max-depth N] [--verbose]`
|
|
|
|
Uses dedicated venv at `~/.local/chamber-tools-venv/` (bs4 + lxml).
|
|
|
|
2. **`pandoc -t gfm-raw_html --wrap=preserve --extract-media=DIR -o OUT.md REPAIRED.epub`** — convert to clean Markdown. The `-raw_html` suffix suppresses HTML scaffolding preservation (the lesson from the Polastrons/Obrist work earlier same day).
|
|
|
|
3. **`~/_Dev/chamber-library/scripts/strip_cruft.py OUT.md --apply --samples 0`** — final cleanup pass for any pandoc class residue + link/image attribute stripping. Writes backup to `~/_Dev/chamber-library/_backups/YYYY-MM-DD/`.
|
|
|
|
**Tested 2026-05-13 on three different publisher conventions, all successful:**
|
|
|
|
- Rowman & Littlefield (Bachelard *Dialectic of Duration*): `<p class="CN">/<p class="CT">` encoding, anchor IDs in NCX. 19/20 promoted; final MD has 19 H1 headings including all 8 chapter titles ("Relaxation and Nothingness", etc.).
|
|
- Routledge (Weil *Gravity and Grace*): chapter-per-file encoding, single styled-title paragraph per file. 39/46 promoted (front matter skipped); final MD has 38 chapter H1 headings.
|
|
- Bloomsbury (Gadamer *Truth and Method*): `<p class="cn">/<p class="ct">` for chapters + `<p class="pn">/<p class="pt">` for parts, no fragments in NCX. 33/35 promoted; final MD has 10 H1 + 16 H2 = 26 headings (3 Parts + 5 main chapters + 6 Appendices + 2 Supplements + Translator's Preface).
|
|
|
|
**Why the tool is publisher-convention-independent:**
|
|
- Uses NCX/NAV (required by EPUB spec) as authoritative TOC source
|
|
- Matches by anchor id (preferred) OR by text content (fallback)
|
|
- Text matching handles leading numeration in many forms: arabic ("1 Title"), Roman ("III: Title"), English words ("Part One Title")
|
|
- Number-stub fall-through: when anchor target is just a chapter number (e.g., "Chapter 1"), falls through to text matching to find the substantive title
|
|
- Inner `<a>` wrapper stripping by default so pandoc produces clean `# Title` not `# [Title](anchor)`
|
|
|
|
**Important context — the failure mode this tool fixes:**
|
|
|
|
Many scholarly EPUBs (especially from typesetting workflows like Adobe InDesign export) encode chapter titles as styled paragraphs:
|
|
```html
|
|
<p class="CN">Chapter 1</p>
|
|
<p class="CT">Relaxation and Nothingness</p>
|
|
```
|
|
rather than:
|
|
```html
|
|
<h1>Chapter 1: Relaxation and Nothingness</h1>
|
|
```
|
|
|
|
When pandoc converts these with default flags, it sees only paragraphs — no headings to promote. Output is structurally flat regardless of the conversion flag. The chamber-library audit flagged 127 files in this state (RECONVERT bucket).
|
|
|
|
This tool fixes the source EPUB first so any subsequent conversion (now or later) inherits proper structure.
|
|
|
|
**Filing discipline (steward's directive 2026-05-13):**
|
|
- **EPUBs and MD conversions live in DIFFERENT locations** — never mix
|
|
- **Repaired EPUBs**: `~/Documents/__Chamber Source EPUBs/repaired-YYYY-MM-DD/<descriptive-name>.epub`
|
|
- File apart from chamber-library entirely; this is source-material territory, not converted-text territory
|
|
- Naming matches the chamber MD filename (e.g., `the_dialectic_of_duration_bachelard.epub` for chamber `the_dialectic_of_duration_bachelard.md`)
|
|
- Double-underscore prefix on the parent folder follows steward's convention for important folders like `__must read/`
|
|
- **MD conversions**: filed at the canonical chamber-library path (e.g., `~/_Dev/chamber-library/converted_texts/traditions/critical_modernity/phenomenology/the_dialectic_of_duration_bachelard.md`)
|
|
- **Original EPUBs** in `~/Documents/___The Library [ePub_AWZ3]/` and `~/Desktop/New texts may 2026/` are never modified
|
|
- **Existing chamber file backups** (when replacing a broken existing MD): `~/_Dev/chamber-library/_backups/YYYY-MM-DD/<rel-path>.preconversion-backup`
|
|
- `mempalace mine` automatically detects mtime change on the chamber MD → deletes old drawers → re-mines clean content on its next iteration through that directory
|
|
|
|
**For the broader RECONVERT queue (127 files):**
|
|
|
|
This tool is the right shape for any EPUB-source RECONVERT file. For PDF-source files, docling is the conversion tool (per the toolset study 2026-05-13). For files where the source can't be found in `~/Documents/___The Library [ePub_AWZ3]/` or `~/Desktop/New texts may 2026/`, the file stays in the chamber as-is until source surfaces.
|