33 lines
3.3 KiB
Markdown
33 lines
3.3 KiB
Markdown
---
|
|
name: Articulate toolchain fit before executing conversion or processing work
|
|
description: Before running any conversion/processing tool, surface why this tool is the right one for this input, what its limits are, and where in the broader toolchain it sits. Universal-tool framing is wrong; tools-by-input-type is right.
|
|
type: feedback
|
|
originSessionId: a5411fd1-4a96-43e7-aa1a-1a50d8951a8f
|
|
---
|
|
When proposing or executing **conversion / processing operations** (EPUB → MD, PDF → MD, OCR, transcription, etc.), do not present a single tool as "the" tool. **Articulate the toolchain fit first.**
|
|
|
|
**Why:** Steward's articulation (2026-05-13): *"I thought that pandoc was the tool for conversion and document creation. I'd like to know more about where the current toolset is truly ideal for my needs. Now you have enough experience with me to know how I work, and, remember that we haven't even begun with the historical treatises… they could prove more difficult to process."*
|
|
|
|
The contamination-aware reading: I had been treating pandoc as a universal tool and docling as a competing alternative. That framing is wrong. They are *complementary*, not substitutes — each excels on a different input type. Surfacing "use X" without surfacing "X is right for input-type Y; for input-type Z we'd use W" leaves the steward unable to think strategically about the corpus they actually have.
|
|
|
|
**How to apply:**
|
|
|
|
Before any conversion run, surface:
|
|
1. **What kind of input is this?** EPUB, PDF (with text-layer / without), HTML, scan, audio, etc.
|
|
2. **What does the input's structure require?** Multilingual? Multi-column? Footnotes? Marginalia? Equations? Historical typography?
|
|
3. **Which tool best matches?** Brief rationale, not just a name. (Pandoc for clean structured inputs; docling for modern PDF layout reconstruction; ocrmypdf for missing text-layer; marker for equation-heavy academic PDFs; etc.)
|
|
4. **What does this tool *not* preserve?** Honest about losses (footnote linking, image fidelity, complex layout).
|
|
5. **Where in the toolchain does this sit?** Is this one stage of a pipeline (e.g., OCR → docling → manual review) or one-shot?
|
|
|
|
For the steward's likely future corpus (historical treatises, scanned classics, multilingual sources, varying-age digital editions), there is no single tool. The right answer is a **diagnosed pipeline per input class**, not a default tool.
|
|
|
|
**Tooling memory anchor:**
|
|
- **pandoc** — universal converter for clean structured formats (EPUB, HTML, LaTeX, Markdown). Excellent multilingual. Fast.
|
|
- **docling** (IBM, pipx) — PDF/DOCX/PPTX layout reconstruction. Does NOT handle EPUB. Heavy install (torch + models). Best for modern academic PDFs with text-layers.
|
|
- **marker** — alternative to docling for PDFs, especially equation-heavy academic papers.
|
|
- **ocrmypdf + tesseract** — for PDFs without text-layers or with poor OCR. Pre-processor for docling/marker.
|
|
- **Calibre's ebook-convert** — handles weird EPUB edge cases that pandoc can't.
|
|
- **Direct text extraction (pdftotext)** — fast but loses all structure; useful as a fallback or for quick inspection.
|
|
|
|
The historical treatises will likely need OCR + docling + manual review per text — not a single automated pass. Treat as research-grade conversion: surface that reality before suggesting a one-line command.
|