79 lines
20 KiB
Markdown
79 lines
20 KiB
Markdown
---
|
||
name: session-ledger-2026-06-29
|
||
description: "Practice-of-return ledger maintained by /symmetria — returns, open horizons, recalibrations, authorization moves, sub-agent dialogues, bypasses."
|
||
metadata:
|
||
node_type: memory
|
||
type: feedback
|
||
originSessionId: 604b3d1d-6111-48f2-9683-2dfba02d5b6b
|
||
---
|
||
|
||
# Session Ledger — 2026-06-29
|
||
|
||
## Returns
|
||
- 2026-06-29T09:40 — Symmetria check before modifying a corpus-wide cleaning tool (blast radius beyond local tree). DIAGNOSIS (class, not instance): footnote conversion is split across two tools, each pinned to ONE publisher anchor convention — clean_pandoc_html_residue (`<sup><a id=…fnN>` HTML; Musil) and clean_epub_residue (`[N](chunk…#nfK)` md-link; Calibre). Taylor is a THIRD shape: `[¹](#notes.html_fn-N)` ref / `[ N.](#chapter…rfn-N)` def — matched by NEITHER. Silent-loss failure mode: the generic internal-link unwrap in clean_pandoc_html_residue EATS unrecognized footnote links → ref becomes bare superscript, def orphaned. The prose word-guard does NOT catch it (footnote labels excluded). THIS is how Taylor's notes were "dropped." Fittingness: surface before rewriting (Honest-degradation constitution clause; silent failure = architectural violation). Confidence 0.92 (measured all three shapes in fresh pandoc output; 59 refs ↔ 59 defs, globally numbered 1–59 = clean 1:1 key). Recommend: generalize the md-link converter's anchor family + run footnote-conversion before generic unwrap. Logged, surfaced to steward, NOT yet applied.
|
||
|
||
- 2026-06-29T10:10 — Taylor (Ethics of Authenticity) converted to canonical-ready, gold gate PASSES (cruft=0, raw_html=0, 31528 words, 705 lines). Authorized tool generalizations applied: clean_epub_residue footnote family B (#…_fn-N/#…_rfn-N, 59↔59 pairing) + IMG_LINE generalized by extension (caught a SECOND instance of the same class-bug: tool pinned to `img/`, Taylor used `images/`) + clean_pandoc_html_residue loud-abort guard (FOOTNOTE_LINK, fired correctly on 118 links). Per-book structure pass kept reproducibly at scratch/making/taylor_structure.py (doubled-heading collapse using the book's OWN ToC casing, front-ToC + page-index drop, drop-cap de-italics — all prose-word-invariant). NOTE: doubled-contents-backlink-headings + front-ToC + page-index = a recurring EPUB shape; promote scratch script to a governed tool IF the next EPUB confirms it (build-on-need).
|
||
|
||
## Open horizons
|
||
- TOMORROW MORNING (steward priority, before 15:00 Seb call; Seb is 6h behind so he's asleep): **PRIORITY 1 — stand up CapableHands as a BMF clasp** (43M tiered-circle-inference), pair the M1 Air as a light member, validate the offload path → solves PENDING-41 (Air saturation) → demo for Seb. Scope: `CapableMind-AI/docs/thinking/David/l1-reliability/clasp-on-capablehands-scope-2026-06-29.md`. KEY FINDINGS: 43M is BUILT in BMF, but CapableHands is GREENFIELD (no Ollama, no BMF, no models — verified) → fresh deploy, not a flag flip. Two genuine unknowns = (a) how claspDuties get assigned [isClasp=claspDuties.size>0, read bootstrap.ts+pairing], (b) the circle-pairing procedure [src/core/pairing/manifest-manager.ts]. Prereq: OCR must be done (free by morning); verify qwen3.5 is a real Ollama tag vs fallback (ask Seb). **PRIORITY 2 — the library metadata indexer** (`chamber-library/_curation/library-indexer-scope-2026-06-29.md`).
|
||
|
||
- 2026-06-29T10:55 — Taylor PUSHED: chamber af17dde..e74ccc8 (skemantix), studium 3b2b521..844b207 (github). Graduation pattern proven end-to-end. NEXT in the batch: cleanest EPUBs (Virilio, Handke, Alter…) + the 126 audit fails — same trustworthy gate, same keep-notes discipline, now with the generalized footnote/image tools.
|
||
- 2026-06-29: BATCH progress — 3 graduated + pushed: Taylor (hermeneutics), Virilio (french_theory), Handke (literature/contemporary, sha 7a83a42b). Heading-de-linkify confirmed 3× (the common reusable atom) but per-book front/back-matter logic differs enough that a full structure-tool would need variation-handling — DEFER promotion until a clean abstraction emerges; per-book scripts archived. NEW GUARD LESSON: the prose word-guard validates only the KEPT region, NOT that the right region was kept — Handke near-miss (a back-matter phrase also lived in the Contents ToC → end-boundary matched too early, dropped the whole memoir, guard still said CLEAN). Added a total-prose size-sanity check (kept ≥70% of source words). Carry this into every structure script.
|
||
- NEEDS A FOCUSED PASS — salvageable, NOT a quick graduation (2026-06-29): Agamben, The Man Without Content. Kindle-filepos EPUB. Body prose CLEAN (no pervasive corruption) BUT: (a) structure must be recovered — Calibre-normalize + repair gives 12/12 positions but EMPTY titles; structure_from_ncx HAS the 12 titles but with number-OCR (§ i→1, § z→2, § io→10, "Notes u9"→119); (b) scattered minor body artifacts (broken words "intellec tual", "signssigns", "com- pletely") that need a careful read to catch all; (c) real art-plates (00014=74k Dürer/gallery, 00022-00025, 00028) needing drop-but-keep-caption decisions; (d) decorative ornaments (tiny) droppable. Toolchain ready (Calibre + repair venv + toc.ncx present). Recommend a focused pass, not rushed at long-session tail.
|
||
- RESOLVED 2026-06-29: Mumford Technics and Civilization — steward surfaced a CLEAN PDF (`~/__Making sequence sources/Technics_and_civilization_Mumford.pdf`, 532pp, 30M) with a CLEAN TEXT LAYER (tlie/modem/ofthe = 0 across spans; uniform 1869-2076 ch/pg; single-column). Supersedes the corrupt EPUB (discard the EPUB). Running docling text-layer pipeline LOCALLY (nohup PID 22340 @ launch, docling 2.92.0, `--to md --output scratch/making/docling`); log `scratch/making/docling-technics.log`. NOT OCR'd — clean text layer beats re-OCR (do-it-once). When done: post-process (drop running heads "TECHNICS AND CIVILIZATION"+page-nums; structure; verify; graduate). [former note kept below]
|
||
- [former, now resolved] PARKED — needs a CLEAN SOURCE: Mumford, Technics and Civilization — the available EPUB is a bad OCR SCAN (letter-substitution corruption: "tlie"×74 = the, "modem"×10 = modern, "distrihiition", "])()vver" = power), zero structural markup, no anchors, + photographic plates. Unsuitable as-is; graduating would inject corrupted text. Needs a clean digital edition. GATE LESSON: verify's ocr_damage check does NOT catch letter-substitution OCR errors (only word-final-& / odd-punct) — a scanned EPUB can pass ocr_damage while badly corrupted; the prose-sample pre-check is the real catch. Candidate hardening: wordfreq/dictionary OCR-error ratio on EPUBs too (normalize_ocr exists for the OCR pipeline; not applied to "clean" EPUBs).
|
||
- PARKED — needs a FOCUSED SESSION (steward 2026-06-29): Alter, The Book of Psalms — glyph→Unicode map (class 1 romanization diacritics: I build+verify visually; class 2 Hebrew-script sequences: with steward). Scratch conversion left at scratch/making/the-book-of-psalms-robert-alter.md (NOT graduated). See [[feedback-character-as-image-hazard]].
|
||
- DEFERRED (steward 2026-06-29): reading-index design — protocol-dependent (chavruta/debate/connection-surfacing), probably plural; do NOT design schema until corpus gold + protocols declare themselves. `.meta.json` structural sidecar is the protocol-neutral standard meanwhile. → [[project-sidecar-typology-protocol-dependent]]. Sidecar standard itself to be reevaluated once corpus verified gold.
|
||
- DONE 2026-06-29: folded RAW_HTML into audit_cruft (commit 9e99e07) — single-source gate, verify imports it, no double-count, monotonic. Canonical true-state now HONEST: 1215 clean / 72 with latent cruft (the backlog — Bachelard 865 <sup>, Alexander Oregon, Bringhurst… real residue, verified not false-positive). The 72 are the existing-fails fix-list, logged, NOT being fixed now (steward chose forward graduations).
|
||
- BATCH (4 graduated + pushed): Taylor (hermeneutics), Virilio (french_theory, family-C), Handke (literature/contemporary), Illich Deschooling (contemporary_voices/environmental, sha 288bb713, no-notes edition, shelved w/ existing Illich). Alter PARKED (character-as-image, focused session). Remaining clean candidates: Mumford (Technics), Agamben (Man Without Content), Calasso (Ruin of Kasch), Han (Transparency / In the Swarm), Borgmann, Crawford (Shop Class). CAUTION-flagged (special, pre-check needed): Sebald (integral photographs), Celan Breathturn (bilingual + diacritics), any non-Latin/diacritic-heavy source. Image pre-check now mandatory per [[feedback-character-as-image-hazard]].
|
||
- (earlier) IN PROGRESS note — Taylor (hermeneutics), Virilio (french_theory, sha a021142a, family-C caret-chapter footnotes added). Both pushed. clean_epub_residue now handles 4 footnote conventions (A chunk / B anchor-id fn-N / C caret-chapter chXfnY). RECURRING TRUTH: every publisher = new footnote convention + bespoke structure; per-book structure scripts archived in scripts/archive/conversions/. Heading-de-linkify is the common reusable op — promote to a governed tool after Handke confirms a 3rd time. NEXT: Handke (A Sorrow Beyond Dreams), Alter (Book of Psalms). Placements (hermeneutics, french_theory) flagged in sidecars for steward redirect.
|
||
|
||
- 2026-06-29T09:16 — PULLING THREAD (confirmed): the graduation batch — settle the rest of the corpus (~18 Making EPUBs + ~9 PDF/OCR + 126 audit fails), each properly (keeping notes), against the now-trustworthy gate. Taylor-redone-WITH-notes = first piece + pattern-setter.
|
||
- Open question gating batch order: new Making graduations vs the 126 existing audit fails — which serves the engine sooner? Sub-q: fold RAW_HTML into audit_cruft (cleaning-pipeline gate, not only verifier).
|
||
- Beside-the-thread (engine phase, gated on corpus): new studium commit 3b2b521 — jurist disposition on the two owed sweeps (genealogy/temporal; multi-voice). Read when turning to the engine, not before.
|
||
- Paper debts owed by steward: REVIEWED-44 (operations.yaml) + REVIEWED-45 (Chamber spec §§I–VI) — both authorized live, paper record outstanding.
|
||
|
||
## Bypasses
|
||
- 2026-06-29: STOPPED before graduating Alter (Book of Psalms) — refused the batch-pipeline because the 356 "images" are character-bearing (Hebrew transliteration diacritics ḥ/ʾ/ʿ + full Hebrew-script words as image-sequences), NOT decorative. Dropping them = silent verbatim corruption (PASS-BUT-FALSELY: gate would pass, text mutilated). Verified by extracting+viewing the glyphs (h.jpg=ḥ, 30a=ʾ, 30b=ʿ, 125a=ה). Parked pending a dedicated glyph→Unicode mapping with the steward (Hebrew script needs his eyes). New standing lesson → [[feedback-character-as-image-hazard]]. This is the prime directive enacted: don't rush dwelling-work; the gate correctly protected the corpus.
|
||
|
||
## Chamber metadata layer is largely INCOMPLETE (steward flagged 2026-06-29, wrap)
|
||
- **Sidecars: 1282 canonical works · 19 .meta.json sidecars → ~1263 MISSING.** Only the recently-graduated have them.
|
||
- **"~1073 without frontmatter" — but NOT a metadata void.** VERIFIED: 952 are Loeb files with NO YAML but a CONSISTENT parseable header (`# AUTHOR, Work` H1 + `*Loeb Classical Library — bilingual (original + English)*` + `**volume**` + `###### — p. N —` page anchors) — produced deliberately by `loeb_chamber_adapter.py`, predates the frontmatter convention. So the metadata IS there (author/work/bilingual-lang/page-provenance), just not as YAML → MECHANICALLY backfillable (parse the H1, OR re-read the richer DSL source file in the library). Split: 952 Loeb (easy, H1-parse) + ~121 genuinely-bare non-Loeb (indexer internal-metadata route) + ~209 already-have-frontmatter (ready). The Loeb voice = the author; lang = grc+en / la+en. **VERIFIED never-stripped, never-YAML** (steward asked "or was it stripped?"): the DSL extraction scripts (extract_loeb_dsl_phases_2_4.py / complete_loeb_extraction_final.py) CONSTRUCT each file as `# {author}` + `**Source**: Loeb…` + `## {work}` markdown — never a YAML block; strip_cruft explicitly PRESERVES frontmatter (NOT-touched list); earliest git commit 24888bc shows the same. No data loss, no destructive bug — predates the frontmatter convention. Backfill = parse existing headers / re-read DSL source.
|
||
- INSIGHT: a sidecar is mostly MECHANICAL (sha/lines/path = computed; work = from slug; sections = heuristic: YAML→apparatus, body→text/citable, `## Notes`→apparatus). The CURATORIAL fields (voice/lang/section-roles/_note) need real metadata → blocked by the no-frontmatter gap.
|
||
- THESE ARE ONE PROGRAM ("complete the chamber metadata layer"), dependency-ordered: (1) frontmatter-backfill [harvest title/author/lang from source ebook INTERNAL metadata = what the indexer reads] → (2) sidecar-generation [mechanical core + backfilled frontmatter; the 19 hand-authored = gold template] → (3) source-link + (4) dedupe [indexer outputs]. → The library metadata indexer (`_curation/library-indexer-scope-2026-06-29.md`) is the KEYSTONE that unblocks all of it. Sidecar gap STRENGTHENS the build-indexer-first plan. See [[project-source-library-link-and-dedupe]].
|
||
|
||
## PDF OCR — CapableHands (launched 2026-06-29 15:15)
|
||
- RUNNING detached on CapableHands (david@10.0.1.136, `~/capablehands/`): `making-ocr-queue.sh` (nohup, PID 9602), olmOCR-7B MLX @ 200dpi, 3 single-column scans SEQUENTIAL (~10h total, ~30s/pg): jonas-imperative-of-responsibility (280pp, FIRST) → mumford-technics-and-human-development [Myth of the Machine vol I] (388pp) → mumford-pentagon-of-power [vol II] (540pp). Output → `scratch/<name>.md` (chamber `## Page N` form). Log: `scratch/making-ocr-queue.log`. STATUS CHECK: `ssh david@10.0.1.136 'tail ~/capablehands/scratch/making-ocr-queue.log; ls -la ~/capablehands/scratch/*.md'`.
|
||
- PDF TRIAGE (8 PDFs in ~/__Making sequence sources): OCR-needed single-col scans = the 3 above (sent). TEXT-LAYER (NO OCR — use docling/pdf_textlayer later): Speed_and_Politics (Virilio, 175pp, 1468 ch/pg), The_Burnout_Society (Han, 69pp), Encyclical Leo XIV (55pp, born-digital — likely out of Making scope). SKIP: Levi drowned-and-saved (already graduated Position I). FLAGGED: Psalms_2 commentary (Hossfeld-Zenger Ps 51-100, 592pp) — single-col but dense scholarly apparatus (bibliography/textual-notes/marginal Selah); possibly a REFERENCE work outside Making scope — confirm before OCR.
|
||
- DOCLING DONE 2026-06-29 21:58 (all 3 text-layer PDFs, in scratch/making/docling/): Technics_and_civilization_Mumford.md (8117 lines, tlie:0 modem:0 CLEAN — supersedes the parked corrupt EPUB ✓), Speed_and_Politics_Paul_Virilio.md (948), The_Burnout_Society_Byung_Chul_Han.md (461). NEED post-processing: `&`→& in Technics heading; letter-spaced display headings ("T H E B U R N O U T") normalize; front/back trim; verify; graduate. Placements: Mumford T&C → critical_modernity or a new node; Virilio Speed&Politics → french_theory (w/ Information Bomb); Han Burnout → critical_modernity (w/ a Han home, cf In the Swarm/Transparency).
|
||
- CapableHands OCR mid-flight at wrap: Jonas 147/280 (~29.5s/pg); then Mumford MoM I (388pp) + II (540pp) overnight (~7.5h). Check: `ssh david@10.0.1.136 'tail ~/capablehands/scratch/making-ocr-queue.log; ls -la ~/capablehands/scratch/*.md'`.
|
||
- NEXT (after OCR completes): quality-check Jonas FIRST (tool-review discipline; olmOCR single-col proven on Levi) → post-process via OCR pipeline (normalize_ocr + page-number structure recovery + clean + verify) → graduate. The Mumford PDFs are "Myth of the Machine" I+II — NOT the parked "Technics and Civilization" EPUB (different work; that still needs a clean source).
|
||
|
||
## Remaining-EPUB triage (2026-06-29) — they are NOT quick clears
|
||
Convert+pre-checked 6 remaining EPUBs. All TEXT-CLEAN (no OCR letter-corruption, no character-images) BUT all Kindle-derived (filepos/id_N/split-node anchors), STRUCTURE-LOST, and unrecoverable by the automated tools (validated on Han Transparency: ebook-convert→repair = 0/15 headings, 0%; structure_from_ncx = 0 chapters). Same class as Agamben → each needs MANUAL heading placement. Footnote forms vary per book (all caret-wrapped `^([N](#...))` but anchors differ: Crawford=filepos393373, Han=id_196, Borgmann=ifm1en1) — family-C regex (Virilio chXfnY) won't match; needs generalizing OR per-book handling.
|
||
- crawford-shop-class (L2528, img8, caret-fn filepos) · steiner-bluebeard (L1937, H1, img20 — pre-check images!, no caret-fn) · han-transparency (L609, caret-fn id_N) · winnicott-playing-and-reality (L1991, no caret-fn) · borgmann-technology-character (L4210, caret-fn ifm1en1) · calasso-ruin-of-kasch (filename was truncated: The_Ruin_of_Kasch_..._z_li.epub — not yet converted).
|
||
RECOMMENDATION: these form a coherent sub-batch ("Kindle-derived, structure-lost"). Worth building ONCE: (a) a ToC/filepos→body structure-recovery tool, (b) generalize caret footnote family to varied anchors. Then clear them together as a focused session. Did NOT rush them at long-session tail (prime directive: don't rush dwelling-work). Triaged .md kept in scratch/making/ (gitignored).
|
||
|
||
## Existing-backlog cleanup — the 70 (mapped + started 2026-06-29)
|
||
The RAW_HTML fold-in revealed 72→70 canonical files with cruft. FULLY TRIAGED by signature + actual tag content:
|
||
- **DONE (commit 9b986c1):** svg/nav safe class — built `clear_safe_html_residue.py` (touches a file only if ALL raw_html ∈ {svg-block, nav-block, emphasis} AND prose-word-guard clean). Cleaned 11 (5 now fully clean → 1220/70). Guard REFUSED 9 (Arden Shakespeare: untight <nav>/</nav> would delete 10,497 words — caught).
|
||
- **fenced_div (23 files)** = pandoc `:::` markers, the BIGGEST bucket (Buddhist Samyutta 62588, Taruskin 25376, typography-for-lawyers 8720, + poetry). NON-poetry → strip_cruft (mechanical, the runbook's lock-step tool). POETRY (Les Fleurs du Mal, Stevens, Duino Elegies, ce-peu-de-bruits, carnets) → JUDGMENT NEEDED: is `:::` verse/stanza structure or residue? Don't bulk-strip poetry blind.
|
||
- **bare-<sup> OCR (9 files):** Bachelard ×3 (poetics-of-reverie 865 — superscripted title letters `THE POETICS <sup>O</sup><sup>F</sup>` + bare `<sup>N</sup>` footnotes + split `"` entities), Alexander Oregon 580, typography books (Bringhurst/Hochuli/Morison/Müller-Brockmann sup-footnotes). Real per-file footnote-conversion + entity-fix work. NO clean <sup><a> cases exist (Taylor tool doesn't apply).
|
||
- **structural-HTML (21 files):** <p>/<figure>/<blockquote>/<a> residue — refactoring (226 <p>+136 figure), Muir (320 <p>), Kafka, Orwell's Roses, Mythologiques-1 (62 blockquote). Signals POOR ORIGINAL CONVERSIONS — likely re-convert, not patch.
|
||
- **repl_char (6 files):** U+FFFD decode LOSS — Arcades Project 2529!, histoire-de-la-folie 246, roman-constitution 81. Data GONE; needs re-derivation/re-source (Mumford-class).
|
||
- **md_image (3 files):** philosophical-writings 290 (!), book-of-me 91 — PRE-CHECK for character-glyphs (Alter-class) before any drop.
|
||
HONEST META: no large "trivial" bucket exists; the backlog is heterogeneous accumulated OCR/conversion debt. Each remaining bucket needs per-file care or judgment. The whole-chamber-settle is bigger than "easy and doable"; consider settling ENGINE-PRIORITY works first (Making/chavruta) over the long tail.
|
||
|
||
## Confidence to recalibrate
|
||
- Carry from last session: drift shape = a finding scoped to one condition restated as an unconditional verdict (olmOCR "retired" from a 2-col-scoped finding). Verify the scope of a claim before generalizing. Kin to trust-prior-pass-frame.
|
||
- Recurring return that paid off: measure, don't theorize (caught the French-mislabel, the Shakespeare verse, the guard delta). Hold today.
|
||
|
||
## Authorization moves
|
||
- 2026-06-29T09:40 — Steward authorized (AskUserQuestion) "generalize the shared converter" for the footnote-conversion gap (Option 1: broaden anchor family + run-before-unwrap + loud-abort guard). [HARDENING-class; applied.]
|
||
- 2026-06-29T10:20 — Steward authorized (AskUserQuestion) placement: NEW tree node critical_modernity/hermeneutics for Taylor (and incoming communitarian-hermeneutic siblings).
|
||
- 2026-06-29T10:40 — Taylor graduation COMPLETE: canonical (sha c8353c65, 705 lines) + .meta.json sidecar (studium) + catalogue rebuilt (1289). All gates pass. Conversion record preserved at scripts/archive/conversions/taylor_ethics_structure.py. Staged, NOT yet committed — awaiting steward go on commit+push (harness: ask before commit/push; chamber precedent = direct-to-main + push).
|
||
|
||
## Sub-agent dialogues
|
||
|
||
## Bypasses
|