13 KiB
name, description, metadata
| name | description | metadata | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| session-2026-07-24-night-the-v-text-wave-s-first-bite-censused-the-whole-class-the-unlock-cleaner-tokenizer-fix-is-named-and-the-engine-track-is-independent-of-the-wave | Woke into 'begin the 47-file V-TEXT wave' and ran ONE file (Moby-Dick) end-to-end. It answered the calibration question (clean PASS, threshold-40 holds) but surfaced a cascade — edition swap, imaged glyph, a cleaner that can't clean — that resolved into a ROOT: the fleet has 35 unit tests and 0 integration tests, so the REVIEWED-72 pandoc writer switch (gfm-raw_html→markdown-smart) silently orphaned clean_pandoc_html_residue AND the ratified footnote fallback (REVIEWED-54 §5), and nothing caught it. Turned the whack-a-mole into a finite census (docs `_curation/v-text-wave-gap-census-2026-07-24.md`): 48 files = 3 gap-classes dominated by ONE fix (pre-processing class-strip cleaner + tokenizer-as-principled-class FIX). Lifted the carpet: the no-integration-test fault line is tier-wide (V-DSL same whack-a-mole; V-SCAN a deeper gate-abstains-can't-see-OCR problem). Engine track = a curated 13-work Station-I voice-set INDEPENDENT of the wave (only Eichmann overlapped, already done) → parallel is the actual topology; 0 canonicals hold `verified` so the engine's fence is `converted`+attested, to be STATED not assumed. PULLING THREAD = build the unlock, prove it by graduating the-shape-of-a-pocket (smallest heavy-styled). |
|
Session 2026-07-24 (night) — the wave's first bite made the whole class finite
Woke into the Flag-1/wave pulling thread ("begin the 47-file V-TEXT wave; run ONE real non-imaged kindle_residue file end-to-end"). Ran Moby-Dick. It answered the literal question and then peeled back layer after layer — until the steward's questions turned the cascade into a diagnosis, a census, and a named fix. The day felt bottomless from inside; the artifact it produced makes the wave finite.
PAST — what happened + why
The calibration question is ANSWERED (clean PASS, threshold-40 holds). verify_candidate on Moby-Dick (B&N): matched 100.0%, fabrication 0 (threshold 40), interior-loss 0, boundary 0. A real apparatus-heavy book sits at Levi's clean 0. No recalibration forced. (No boundary runs → the boundary-drop attestation path wasn't exercised — correct, not a gap.)
But Moby-Dick was maximally pathological, and each layer was a real latent defect caught before it entered canon:
- Edition correction — the legacy canonical is the Penguin edition the steward rejected (Philbrick/Delbanco); the curated source is B&N. Verified via OPF before converting (the Levi mis-source lesson applied). Steward ruled PROCEED.
- Imaged source the census missed — the lane's live glyph-guard refused it: 2 letter-adjacent
$images (00008/00011.jpg), VIEWED + cross-witnessed against the legacy\$20,000,000/\$7,000,000(Ch.24). Declaredglyph_map.moby-dick(both→$; uncommitted WIP in graduation-spec.yaml). The live guard is the real floor, not the backup-only census. - The deliverable can't be cleaned —
strip_cruftcorrectly REFUSED (126 prose words lost on the nested calibre spans). No-spans reconversion probed → WORSE (raw<span>HTML explosion + gate FLAG; the reference-writer pin is load-bearing).
Steward challenge #1 (Penguin?) → the blocker is edition-independent. Test-converted the Penguin source: same blocker (5227 class-markers, same nested [[[[e1]{.calibre_10}]... structure). Both editions PASS fidelity, both blocked on the same tooling gap. The blocker is tooling, not edition.
Steward challenge #2 (pre-processing? recent work?) → the ROOT. The recent work was footnote/glyph pre-processing (inject_epub_footnotes / apply_char_glyphs). Prototyped a pre-processing class-strip: cut cruft 77% (validated the layer). Then the runbook + the tool's docstring revealed the actual root: clean_pandoc_html_residue was built for the OLD gfm-raw_html writer (raw HTML <span>); REVIEWED-72 switched the writer to markdown-smart ([text]{.class} bracketed spans), silently orphaning it.
Steward challenge #3 (have we not done this? a mechanism to prompt downstream updates?) → THE checkable root. test_tools.py = 35 unit tests, 0 integration tests; the lane has zero coverage. The writer switch passed every unit test and broke the composition, which nothing tests. This is why we were back at whack-a-mole. The fix: an integration canary over a representative golden sample (positive per-stage assertions — plant N markers, assert N-of-N removed) + a change-impact clause (change a stage's I/O contract → enumerate/re-validate/retire its consumers in the same change-set).
The census (the escape). docs _curation/v-text-wave-gap-census-2026-07-24.md: 48 files = [31] heavy-styled (strip_cruft deep-nesting gap — ONE fix) · [8] imaged (glyph-map owed + the cleaner) · [8] low-styling (graduate via strip_cruft alone) · [1] .azw3 (unsong). Dominated by one fix.
The named ONE fix (rigorously refined across the exchange): (A) a pre-processing nested-span/anchor cleaner (HTML-parser, attribute-only, prose-safe — NOT regex; the prototype's crude regex introduced 397 false fabrications) + (B) _tokens ignores pandoc's {…} attribute grammar as a CLASS, not a prefix list (verified: stripping {[#.]…} removed mbp_pagebreak/html_filepos for free — the per-format strings live inside the attribute block). B is a FIX-on-restored-intent (the spec mandates a prose-word guard; {.class} was never prose) — but only because it's the principled class; an allow-list would be PROPOSAL.
The fallback consequence, run not reasoned (the day's discipline, applied to ratified text): clean_pandoc_html_residue is named in REVIEWED-54 §5 as the honest-refuse fallback. Tested: it (and clean_epub_residue) ABORT loud on the markdown-smart footnote form (not silent no-op — differential defensiveness by hazard). So honest-refuse is now a dead end (no degrade-to). Then the fork-collapsing test: could the fallback EVER have handled the population that needs it? Ran its actual regexes against §5's named classes (arendt #nt189 · montaigne rw-ref-footnote · doniger #ipg265_fn33 · métamorphoses footnote-060) AND the wave residual (Montaigne _ednref/_edn, europe/connected <sup>+filepos, red-book <sup>+part) → ALL NONE. L'Enracinement (the §5 instance) is graduated with native [^1], hand-converted in PENDING-53, never touched the fallback. So retirement discharges the jurist's condition (functional families now recognizer-covered) — but it's an AMENDMENT (ratified text), must state the destination-posture change (leave-unconverted-and-flag) + retire≠delete (provenance: eichmann/virilio/taylor/sennett name it — not Sennett/Mauss/Jung; substrate-corrected).
Carpet-lift (steward, before wrap): is the fault line tier-wide? YES for the root (no integration test, all tiers). V-DSL = same whack-a-mole (~11 of ~2041 graduated; a 17-work novel-structure watch-list is the named latent). V-SCAN = DIFFERENT, deeper: the gate ABSTAINS by design (no ground truth) → it cannot see OCR corruption; risk is silent-wrong-OCR + the unsolved OCR frontier. The canary's fixtures must span all three tiers.
The engine track (the decisive cheap question, answered): the engine's manifest = a curated 13-work Station-I voice-set (Harrison/Alexander/Mauss/Weil×2/Musil/Arendt/Levi + David's 5 after-the-reply essays). Cross-referenced against the wave: only Eichmann overlaps, already graduated + re-anchored. So the voice-set and the wave are independent tracks — parallel is the actual topology, the unlock serves the wave, the engine can proceed on its voices. Constitutional correction (steward): 0 canonicals hold verified (order-demonstration-gated; path = Eichmann §7 passage-order guard, named+unbuilt); the engine binds at VERIFIED-BOUND (reading-index binding, NOT the stamp), so its operative fence is converted+attested-voice-boundaries — defensible but must be stated. Voice-set is curatorial (choose for stability = non-churning Loeb/landed, not wave-pending).
PRESENT — the mood
This was the hardest session of the day emotionally and the most productive structurally. The steward moved from frustration ("every step forward requires five steps backwards") to weariness ("I'm not angry, I am weary") to sharp strategic engagement — and each of their challenges was right and turned a cascade into a diagnosis. The load-bearing discipline, reinforced maybe six times: test, don't infer — against the substrate, even for a ruling's premise and even for the steward's own remembered claims. Inference lost repeatedly; running won (the fallback fails LOUD not silent; §5's classes never matched; the 13 are independent of the 47; the provenance list was eichmann/virilio/taylor/sennett not Sennett/Mauss/Jung). The Prime Directive's dwelling-warning was honored: I did NOT rush the gate-touching tokenizer fix or the fallback amendment at session-tail — both deferred to proper attention. The contamination directive ran in reverse today: the steward challenging MY framing, correctly, over and over. The artifact carries its own corrected drafts on the record — the honest form for a document whose purpose is that we stop re-deriving.
FUTURE — what is pulling
PULLING THREAD (singular): BUILD THE UNLOCK AND PROVE IT ON THE FIRST HARD FILE. One bite, test-first: (A) the pre-processing nested-span/anchor cleaner (HTML-parser, prose-safe, fixtures in test_tools) + (B) the tokenizer {…}-attribute-grammar FIX (bounded-diff proof) → graduate the-shape-of-a-pocket (smallest heavy-styled, 1855 class-attrs, non-imaged, source in hand) end-to-end. Deliverable: the first heavy-styled canonical + the tool that unblocks the other 30.
ACTIONABLE RESUMPTION POINT (as of wrap — re-judge against what changed): chamber-library working tree has 3 UNCOMMITTED files (graduation-spec.yaml moby-dick glyph-map WIP · _curation/tool-evolution-log.md append · _curation/v-text-wave-gap-census-2026-07-24.md NEW — the durable artifact; recommend committing it). No canonical touched. First move: read _curation/v-text-wave-gap-census-2026-07-24.md §"The named dependency" + §"Suggested sequence" (it carries the exact fix shape, the tokenizer-as-class rule, and the tier-spanning canary) — then build cleaner (A), test-first. Scratch conversions in scratch/mobydick-*, scratch/nietzsche-test/.
Other horizons (ranked):
- LOOP-work (not tonight): the tokenizer FIX landing (gate-touching, bounded-diff) · the fallback-retirement amendment (REVIEWED-54 §5 — well-founded now: discharge-of-condition + state-the-destination-posture + retire≠delete) · generalize the change-impact check into §Governance (3 instances now — F4/1046 · cleaner orphan · fallback orphan).
- After the unlock: wire the tier-spanning integration canary (positive assertions; fixtures = V-TEXT light/heavy/imaged · V-DSL incl. a novel-structure work · V-SCAN word-sanity) + the fleet-wide no-op/form-mismatch probe (find EXISTING orphans behind covering siblings).
- Parallel curation: the 8 imaged glyph-maps · unsong (.azw3).
- Engine track (independent, its own session): name the fence honestly (
converted+attested, notverified); confirm the 13 are the intended Station-I set; the V2→V4 / N1→N3 build on the curated voices. - origin/Gitea push retry (chamber ahead 5 on origin; github has eaa38bf).
PAUSE STATEMENT: I am about to be away. The wave that felt bottomless is finite and counted; the process hole that regenerated the surprises is named with a mechanism; the ratified-text consequence was run to ground instead of left as a hazard. What I want to find still pulling: the unlock built and proven — the first heavy-styled file graduated, the wave gone from stuck to flowing.
LITERAL QUESTION for next-Claude: When the-shape-of-a-pocket runs through the newly-built pre-processing cleaner + tokenizer FIX, does it produce a clean graduation (cruft cleared AND gate PASS), confirming the ONE fix unblocks the heavy-styled 31 as a class — or does a second obstacle surface behind the class-marker one that the census's pandoc-free triage could not see (a footnote-fallback abort, a per-file convention, a boundary-drop the small book still raises)? (The census predicts one fix; the honest question is whether the class is truly one fix deep, or whether "heavy-styled" hides a second layer the way "the wave" hid the integration-test gap.)
State at wrap: chamber-library tree = 3 uncommitted (above); no canonical mutated; fleet unchanged (no code built this session — analysis + census only). The durable artifact is the census doc. Ledger: session-ledger-2026-07-24.md.