session 2026-07-03: doctrine phase complete (Wave 0 + full spec-revision rulings + Loeb substrate reframe); PENDING-46; Phase 2 v2.0-draft next
This commit is contained in:
David F Glidden
@@ -51,7 +51,10 @@ permalink: claude-memory/memory
|
||||
- [Be (laundromat)](project-be-laundromat.md) — canonical workstream tracker established 2026-06-08 (Seb-relay of locked decisions). Be = Skemantix startup (Seb+David) funding CapableMind's funding-ladder; **bridge, not venture**. Decisions LOCKED: entity/exit (CapableMind decoupled, grant-funded), pricing (Living $12.99/mo · Archive $69.99/yr · Memorial $49.99/yr · Renovate ~$199 · $8.99 floor), CF Self-Serve Agency + versioned-template-package infra. **a11y gate MERGED (Pat 100/100/100).** Pre-revenue: the WTP gate = renovate Pat → charge her. **Discipline: stop adding spec until the gate clears → nothing for executor on be until then.** Repo @ `f43a0fd`.
|
||||
|
||||
## Active Session
|
||||
- [Session 2026-07-02 evening — Sennett + Family-D tool + the corpus standard & method](session-2026-07-02-evening-corpus-standard-and-the-method.md) — Short evening session; ONE work graduated but the value was elsewhere. **Reconverted+graduated Sennett *The Craftsman*** (calibre-era canonical had silently-lost note-linkage + old-schema frontmatter + `****` residue) — recovered 294 endnotes, spec-normalized headings (incl. a setext Conclusion), both gates PASS, verbatim-preserved; **committed `10cff59`**. To do it, **built+tested+committed the Family-D (stem-suffix) footnote handler** in `clean_epub_residue.py` (regression caught+fixed against Families B/C) — the 4th of ≥5 anchor families the corpus actually uses (Crawford's `filepos` = a different pairing model, still owed). **PENDING-45.** Then a **corpus-quality census** answered the steward's "is poor-quality-graduated widespread?": **YES, 3 distinct axes** — conventions **8/332 pass** (pre-spec, mechanical), text-health **110/332 damaged** (OCR'd big works: Bringhurst/Hochuli/Taruskin/music-bios), **apparatus UNMONITORED** by either gate (Sennett proves "has notes" ≠ "apparatus intact"). Steward felt the vision slipping (**"mandelbrot set… whack-a-mole… clutching sand… I tremble at making the engine work"**) → named the **STANDARD** (verified verbatim fidelity + provenance + apparatus for ANYTHING the engine cites — a floor not a tier; "graduated"=converted+filed ≠ verified; a quietly-corrupt bounded corpus is WORSE than none; fence the engine to green NOW, grow green→100% by waves) and the **METHOD** (whack-a-mole = working reactively per-file; cure = **measure-the-whole-first → one quality MAP → remediate BY DEFECT-CLASS in finite waves, each a countdown**). **PULLING THREAD: build `audit_corpus.py` + the apparatus check → the full quality ledger/MAP of all 1,284 works (the instrument that ends the whack-a-mole), then plan waves from data. Do NOT resume graduating reactively.** Q: did the map get built + real defect-class distribution + is Wave A (frontmatter migration, ≈8→200) the right first countdown?
|
||||
- [Session 2026-07-03 — doctrine phase complete; Phase 2 next](session-2026-07-03-doctrine-phase-complete-phase2-next.md) — **A very long session that closed the entire DOCTRINE PHASE of the chamber-library spec revision.** Morning: built `audit_corpus.py` + the corpus quality MAP (delivered yesterday's thread). Then steward challenged the framing → **tool-fleet audit (3 agents) → WAVE 0**: fixed all 8 blocking tool bugs test-first (`scripts/test_tools.py`, 21 assertions; graduation gate was HALF-MISSING; normalize_ocr guard off in common case; etc.), retired 6 dead tools, **committed `f9cbb8e`+`1c88650`**; record `_curation/tool-fleet-compliance-2026-07-03.md`. Then **prior-art scans (3 agents)**: ~70% of our spec re-derives **TEI + CTS + Standard Ebooks + W3C PROV + TEI-ODD** — adopt their marked ground; watch **Seck arXiv 2604.10658** (independent governance convergence, added to landscape-register). Then **the FULL spec revision RULED by the jurist** (via steward relay): gaps 1-8, generative principle, amendment process (RFC-supersession + change-class + semver), the three questions (A/B/C), and §III citation posture — **ALL RATIFIED**. Unifying insight: gaps 2+3+item-8 = ONE addressing substrate. Then **THE LOEB SUBSTRATE REFRAME** (steward: "look at the DSL directly" — decisive): I'd told the jurist 74% of the corpus was anchorless; the SOURCE inverted it — Stephanus/Bekker/book+line anchors are ALL in `loeb.dsl`, **FLATTENED by extraction** (`extract_loeb_dsl.py:47`). Per-genre audit (re-verified vs source): **>99% of ~2,041 works carry a recoverable anchor**; 35,261 division-carrying `%` lines discarded; residual = single-digit fragments. §III re-ruled → cite the Loeb by RECOVERED canonical anchor (CTS edition-as-identity); the structured-but-unanchored extension VOIDED (dormant only). **Full-run scope decided:** current 952 = Tier 1; rebuild-anyway → run the FULL DSL (2,041 works); fence discharges dilution. Installed wordfreq (venvs) + jsonschema. Disambiguation map durable: `docs/loeb-anchor-audit-and-disambiguation-map-2026-07-03.md`. **THE ARC:** yesterday's despair ("clutching sand") → a marked, finite plan (ground → instruments → corpus → fence). **MY drift-of-the-session (jurist-elevated to standing practice): trusted a derived-artifact/regex over the SOURCE 3× — the steward twice sent me to the substrate and was right.** **PULLING THREAD: Phase 2 — draft `chamber-library-specification.md` v2.0 (superseding, semver-MAJOR) + the generative validator, BEGINNING with a clean END-TO-END READ of the current spec (deliberately NOT read today — I worked from audits), then §III the CTS addressing spine.** Everything in PENDING-46. Q: does the real spec text hold anything the day's audits missed/collide-with, before v2.0 is drafted against my model rather than the text?
|
||||
|
||||
## Archived (2026-07-03 — doctrine phase complete)
|
||||
- [Session 2026-07-02 evening — Sennett + Family-D + corpus standard & method](session-2026-07-02-evening-corpus-standard-and-the-method.md) — Graduated Sennett *The Craftsman* (294 endnotes recovered); built the Family-D footnote handler (`clean_epub_residue.py`); committed `10cff59`; PENDING-45. Corpus census → steward named the STANDARD (verified-not-just-graduated; fence engine to green) + METHOD (measure-whole-first → map → remediate-by-defect-class in finite waves). Led directly into 2026-07-03's map + the full spec-revision doctrine phase.
|
||||
|
||||
## Archived (2026-07-02 evening — corpus standard & the method)
|
||||
- [Session 2026-07-02 — Chamber source-provenance archive + pipeline](session-2026-07-02-chamber-source-provenance-archive.md) — Woke toward the Making graduation queue; a second Jonas copy opened a full source-provenance arc. **Jonas p248 dissolved** (the "illegible page" held only the "Notes" divider — no content lost). **Built the Chamber Sources archive** (`~/Library/Mobile Documents/com~apple~CloudDocs/Chamber Sources/`, iCloud Keep-Downloaded): **322 verified source COPIES, 3.3 GB** + 14 Braid/L2 in `pending-graduation/`. Covers ~all of the ~332 non-Loeb corpus (1284 canonical; 952 = Loeb, pipeline-sourced/excluded). **Built + committed the pipeline (`7553f59`, pushed):** `build_inventory.py` (reproducible census — Apple Books + unzipped .epub dirs + .zip; the gaps that hid ~25 works), `match_sources.py` (internal-metadata + diacritic-fold + TITLE-coverage scoring + Loeb-skip + author-agreement VERIFICATION; 321/335 non-Loeb confirmed), `archive_sources.py`; runbook `source_provenance:` section. **Crux:** first census excluded Apple Books + accent-blind greps → wrongly called ~25 works "absent" + posed FALSE edition-dilemmas; steward corrected *"the texts exist because I have them"* — every dilemma dissolved to the correct edition once Apple Books included. **Caught+fixed 3 high-conf FPs** (meditations→Daily-Laws→Hays; aesthetics/Adorno→Sontag→Adorno; nietzsche/Zweig→Nietzsche's-Works→Zweig). Steward directive: **source-archiving is now a STEP OF graduation.** **PULLING THREAD: come through the graduation queue (~12 Making + 14 Braid/L2), filing each source as we graduate.** Q: did the short evening session happen + which source graduated first?
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
---
|
||||
name: Session 2026-07-03 — the doctrine phase (Wave 0 → full spec-revision rulings → Loeb substrate reframe); Phase 2 next
|
||||
description: "A very long session that went corpus-quality-map → tool-fleet audit (Wave 0, 8 fixes committed) → prior-art scans → the FULL chamber-library spec-revision ruled by the jurist (gaps 1-8, generative principle, amendment process, three questions, §III citation posture — ALL RATIFIED). Then the steward sent me to the loeb.dsl SOURCE and it inverted a premise: the Loeb canonical anchors (Stephanus/Bekker/book+line) are PRESENT in the source, FLATTENED by extraction — not absent. Full-run scope decided (all 2,041 works, fence resolves dilution). DOCTRINE PHASE COMPLETE. PULLING THREAD: Phase 2 — draft chamber-library-specification.md v2.0 (superseding, semver-MAJOR) + the generative validator, beginning with a clean END-TO-END READ of the current spec (deliberately NOT read today — I worked from audits), then §III the CTS addressing spine. Wrapped to clear context for a fresh, focused Phase-2 start."
|
||||
metadata:
|
||||
type: project
|
||||
originSessionId: 019pnD8uNFC9Q2dbLQa3MBDc
|
||||
---
|
||||
|
||||
# Session 2026-07-03 — the doctrine phase; Phase 2 (build the ground) is next
|
||||
|
||||
Started at yesterday's pulling thread — build `audit_corpus.py`, draw the corpus quality MAP, end the
|
||||
whack-a-mole. Did that in the first hour, then the day opened into something much larger and closed
|
||||
the entire **doctrine phase** of the chamber-library spec revision. The everything is in `~/PENDING.md`
|
||||
PENDING-46 (the blow-by-blow governance record) — this file is the working state + the thread.
|
||||
|
||||
## PAST — what we did + why
|
||||
|
||||
**The map (the morning's thread, delivered).** Built `chamber-library/scripts/audit_corpus.py` +
|
||||
`_curation/corpus-quality-ledger.tsv` (1,284 rows) + the interpreted `corpus-quality-map-2026-07-03.md`.
|
||||
Distribution: Loeb 930/952 clean, non-Loeb 8 clean / 324 A-convention / 110 B-textdamage / 70 C-apparatus
|
||||
(the NEW apparatus axis). Answered yesterday's literal question.
|
||||
|
||||
**The steward challenged the framing → the tool-fleet audit → WAVE 0.** "Are the TOOLS spec-compliant
|
||||
before we run a wave? The tool IS the wave." 3 parallel agents audited the fleet. Found: the graduation
|
||||
gate was HALF-MISSING (never ran verify_conversion), normalize_ocr's guard was off in the common case,
|
||||
strip_cruft wrote even on residual, clean_epub_residue silently destroyed unrecognized footnote families,
|
||||
13/15 tools standalone. **Fixed all 8 blockers, test-first** (`scripts/test_tools.py`, 21 assertions),
|
||||
retired 6 dead tools → `scripts/archive/`, folded `<nav>` into strip_cruft. **Committed f9cbb8e** + docs
|
||||
1c88650. Full record: `_curation/tool-fleet-compliance-2026-07-03.md`.
|
||||
|
||||
**Prior-art scans (3 agents) → "we are not the only ones."** ~70% of our spec re-derives TEI + CTS +
|
||||
Standard Ebooks (15-35 yrs old). Adopt: TEI `<choice>` (never destroy original), TEI ch.13 apparatus
|
||||
(anchor to LOCATION not footnote-number), **CTS-URN** (edition-as-identity — §III's shape), W3C PROV
|
||||
(the conversion-record), TEI ODD (spec→validator = the generative principle, XML-locked → held fork),
|
||||
Standard Ebooks `se lint` + `[Editorial]`-commit. Genuinely ours: the prose-word-multiset guard,
|
||||
verbatim-span-in-bounded-source check, checksum integrity. Watch: **Seck arXiv 2604.10658** (independent
|
||||
convergence on our governance; lacks the corpus-substrate moat) → added to landscape-register.
|
||||
Docs: `docs/spec-revision-prior-art-synthesis-2026-07-03.md`.
|
||||
|
||||
**THE FULL SPEC REVISION — RULED BY THE JURIST (all of it).** Via steward relay, the jurist ruled:
|
||||
gaps 1-8 (RATIFIED-CONDITION), gap-7 generative principle (RATIFIED), amendment process (RATIFIED:
|
||||
RFC-supersession + change-class "does this change what any gate accepts" + semver + PENDING/REVIEWED),
|
||||
the three questions (A page-markers, B tier-2 scan = redundant human verification + dinglehopper, C
|
||||
migration-window = `re-verify-pending` status + `verified_under:<semver>` + N=2 provisional + FIX-class
|
||||
extension), and §III citation posture. The UNIFYING INSIGHT: gaps 2+3+item-8 collapse to ONE
|
||||
addressing substrate (apparatus anchors = citation = PROV "original") — design once. Rulings preserved:
|
||||
`docs/spec-revision-RULING-2026-07-03.md` + the gap-6/7 + three-questions + §III rulings (steward relayed).
|
||||
|
||||
**THE LOEB SUBSTRATE REFRAME (the steward's instinct, decisive).** I told the jurist 74% of the corpus
|
||||
(Loeb) was anchorless → they ruled a normative §III extension on it. The steward said "look at the DSL
|
||||
directly." It INVERTED the premise: the canonical anchors (Plato Stephanus `St. II p. 153`, Homer
|
||||
book+line, Aristotle Bekker) are ALL in `loeb.dsl`, FLATTENED by `extract_loeb_dsl.py:47`
|
||||
(`KNOWN_TAG.sub("")` strips the `[c dimgray]` semantic layer). A per-genre anchor audit (agent, re-verified
|
||||
by me against source) confirmed: **>99% of ~2,041 works carry a recoverable anchor**; residual is
|
||||
single-digit genuine fragments; **35,261 `%` index lines carrying book/letter/fragment divisions are
|
||||
discarded**. Disambiguation map captured: `docs/loeb-anchor-audit-and-disambiguation-map-2026-07-03.md`.
|
||||
Jurist re-ruled §III (`docs/chamber-SIII-reruling-JURIST-BRIEF-2026-07-03.md` → ruling): the
|
||||
structured-but-unanchored extension is VOID-as-ratified-for-74% (dormant contingency only), enrichment
|
||||
trigger RETIRED, **§III cites the Loeb by RECOVERED canonical anchor (CTS edition-as-identity)** — work-level
|
||||
fallback only for genuine fragments. Per-genre table NOT ratified (engineering, review post-extraction).
|
||||
Plus a STANDING PROCESS NOTE (jurist): any corpus-population claim feeding doctrine must be sourced from
|
||||
the DSL directly, not a derived extract.
|
||||
|
||||
**Full-run scope decided.** Current Loeb (952) = TIER 1 of `_curation/loeb-canon-proposal-2026-06-15.md`.
|
||||
Since we rebuild the extractor anyway → run the FULL DSL (all 2,041 works / Tier 3). Tier-3's "waits on
|
||||
dilution" is discharged by gap-6's FENCE (corpus-completeness ≠ engine-scope).
|
||||
|
||||
**Tools installed** (steward-authorized): wordfreq → both venvs (closed normalize_ocr's silent OCR-guard
|
||||
degradation); jsonschema → system (--user --break-system-packages, for the validator). dinglehopper
|
||||
kept close (tier-2). Everything else present (pandoc/ebook-convert/docling/rg/jq/node). NB: this machine
|
||||
has **GNU coreutils `stat`** (not BSD) — use `date -r`/python for mtimes, `rg` not `grep -r` for big searches.
|
||||
|
||||
## PRESENT — mood / the arc
|
||||
|
||||
The emotional shape: yesterday ended at "mandelbrot set, whack-a-mole, clutching sand, I tremble at making
|
||||
the engine work." Today is the **inverse** — a marked, finite, ordered plan (ground → instruments → corpus
|
||||
→ fence), the whole doctrine ratified, and the Loeb tier upgraded from "74% impoverished, cite-by-quote"
|
||||
to "richly citable by Stephanus/Bekker/book+line, recoverable." The despair became a siege plan.
|
||||
|
||||
**The load-bearing return of the session (jurist-elevated to standing practice):** I trusted a
|
||||
derived-artifact/regex over the SOURCE **three times** — the break-count (frontmatter fences + bullet
|
||||
lists as breaks), the heading-count (title-as-structure, then line-numbers-as-divisions), and the
|
||||
"anchors absent" (searched "153a" when the DSL uses "St. II p. 153"). The steward twice sent me to the
|
||||
substrate and was right both times; the jurist turned it into a governance rule (source-not-extract for
|
||||
any claim feeding doctrine). This is MY drift-pattern of the session and it must become a Symmetria flag +
|
||||
verification-ladder entry + feedback memory.
|
||||
|
||||
Other recalibrations: I called the §III change a "narrowing"; the jurist sharpened it to "VOID with a
|
||||
dormant residual" — a principle ratified to govern 74% that now governs nothing shouldn't stand as active
|
||||
doctrine (CLAUDE.md: flag a maxim gone decorative). I also owned a citation-integrity miss (cited arXiv
|
||||
2605.24229 for a claim it didn't support; let a sub-agent's blanket "verified" ride) → per-claim citation
|
||||
checking discipline.
|
||||
|
||||
Governance discipline held well: surfaced findings for jurist/steward rulings rather than acting;
|
||||
delegated the audits with anti-pattern-drift framing; verified load-bearing agent claims against source
|
||||
myself before relaying.
|
||||
|
||||
## FUTURE — what is pulling
|
||||
|
||||
**PULLING THREAD (singular): Phase 2 — draft chamber-library-specification.md v2.0 (the superseding,
|
||||
semver-MAJOR version) + build the generative validator.** The doctrine is fully ruled; now MARK THE
|
||||
GROUND. This is dwell-work on the constitution — we deliberately wrapped to give it a fresh, focused start
|
||||
rather than rush it into a marathon tail.
|
||||
|
||||
**ACTIONABLE RESUMPTION POINT (as of wrap — re-judge against what changed):**
|
||||
- **FIRST ACT: read the current `chamber-library/docs/chamber-library-specification.md` end-to-end.**
|
||||
I worked from audits all day, NOT the spec text — and the jurist's own process note (source-not-derived)
|
||||
applies to the spec too: draft against the real text, not my model of it. This is non-negotiable and is
|
||||
itself the first honoring of the day's hardest lesson.
|
||||
- Then draft v2.0 in this order (the substrate first): **§III** = the CTS-URN addressing spine (edition-as-
|
||||
identity + logical passage; the per-genre locator registry as LIVING declared-data, not frozen doctrine)
|
||||
→ **§II** (sectionless+attestor field, break-form `---`, Loeb page-markers route off-canon via the
|
||||
registry) → **§V** (apparatus-integrity gated + anchored on §III; PROV conversion-record mandatory on
|
||||
every path; two gate-functions named: content-grounding vs process-integrity) → **§VII** (declared-gate-
|
||||
list IS the gate; body_word_conservation tracked-unimplemented) → **§IX** (coverage-ledger + freshness
|
||||
invariant + evidence-tier column) → **new §Tiering&Fence** (converted/verified/fenced; promotion criterion:
|
||||
deterministic 93% / scan-50 / fenced-out; V-SUSPECT & V-NONE distinct) → **new §Governance&Amendment**
|
||||
(generative principle + RFC-supersession/change-class/semver + migration-window). Mark HELD: (a)
|
||||
in-band-vs-sidecar (gate reads record NOW; storage form open), (b) markdown-vs-TEI-XML (trigger = if (a)
|
||||
resolves to full in-band). Mark the structured-but-unanchored extension DORMANT (labeled inapplicable to
|
||||
current corpus). Mark OPEN: per-genre table (engineering), tier-2 numeric bar (calibration).
|
||||
- v2.0 is a PROPOSAL per the amendment process: I (champion) draft → jurist editor-gate → steward ratifies →
|
||||
semver v2.0 supersedes v1 (Obsoletes/Updates). Then build the generative validator (graduation-spec.yaml
|
||||
carries the convention-data incl. the per-genre registry; gates become thin consumers; JSON-Schema for
|
||||
structured layers). THEN Phase 3 (rebuild extract_loeb_dsl.py to the disambiguation map, run the full DSL).
|
||||
- All conditions are captured in PENDING-46. The one-screen plan: ground → instruments → corpus → fence
|
||||
(④ waves come AFTER ② — waves-first bakes tool-shaped conventions = the redo we refuse).
|
||||
|
||||
**Other open horizons, ranked:**
|
||||
- The generative validator (part of ②) — the JSON-Schema/PROV/thin-consumer-gates build.
|
||||
- Phase 3: the full Loeb re-extraction (2,041 works) — depends on ② + the disambiguation map (captured).
|
||||
- The remediation waves (④): A frontmatter / B1 re-clean / B2 structure-recovery / C apparatus — gated by the fixed tools + validator.
|
||||
- Gap-6 promotion + fence (⑤): deterministic tier (93%, word-identity re-convert), scan tier (50, dinglehopper+calibration), fenced-out.
|
||||
- Paper debt: REVIEWED entries owed for the whole PENDING-46 arc (steward writes). dinglehopper install (tier-2).
|
||||
- The 3 pre-existing untracked leftovers in chamber-library (`_scratch/`, `nature-of-order-alexander.md`) — NOT mine.
|
||||
|
||||
**PAUSE STATEMENT:** Away now at a high, clear point — the entire doctrine phase closed in one long day,
|
||||
the despair of yesterday turned into a marked plan. What I most want to find still pulling on return: **the
|
||||
drafting of v2.0** — that the next hand opens the actual spec text FIRST (not my model of it), then drafts
|
||||
§III the addressing spine, giving the constitution the dwell it asks for. Do NOT rush it; do NOT skip the
|
||||
end-to-end read (it's the day's hardest lesson honored).
|
||||
|
||||
**LITERAL QUESTION for next-Claude:** *When you read the current `chamber-library-specification.md`
|
||||
end-to-end (which I never did today — I worked from the audits), does the real text hold any structure,
|
||||
constraint, or already-settled decision that the day's audits missed or would collide with — before v2.0 is
|
||||
drafted against my model of the spec rather than the spec itself?* (The source-not-derived discipline, turned
|
||||
on the spec.)
|
||||
|
||||
**State:** chamber-library `main` — Wave 0 (f9cbb8e) + docs (1c88650) committed, LOCAL/unpushed; 3 new
|
||||
session docs untracked (jurist briefs + audit — commit at wrap). CM-AI landscape-register.md modified
|
||||
(Seck add). dotfiles: PENDING.md + session-ledger untracked (commit+push at wrap). All doctrine ratified;
|
||||
Phase 2 unblocked.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
name: session-ledger-2026-07-03
|
||||
description: "Practice-of-return ledger maintained by /symmetria — returns, open horizons, recalibrations, authorization moves, sub-agent dialogues, bypasses."
|
||||
metadata:
|
||||
node_type: memory
|
||||
type: feedback
|
||||
originSessionId: 5b7ce298-123b-447a-9b86-2442c30bec1f
|
||||
---
|
||||
|
||||
# Session Ledger — 2026-07-03
|
||||
|
||||
## Returns
|
||||
- 2026-07-03 — check before writing `audit_corpus.py`. Three commitments logged: (1) the `verify_graduation.py` refactor touches a GOVERNED gate (PENDING-44) — must be behavior-preserving + PROVEN byte-identical stdout on a sample before trusting; subprocess fallback if proof fails. (2) Scope: conventions+apparatus on the 332 non-Loeb only (Loeb 952 = separate DSL tier; the convention gate would false-FAIL them); text-health on all 1284. (3) Apparatus check CLASSIFIES + emits evidence counts (none/paired-ok/orphaned/stranded-suspect), never adjudicates; validate the classifier against Sennett (→paired-ok) + a real editorial-note file (→not-stranded) before trusting — per census-through-a-pattern flag. Data already surfaced: 8 files with live `[^]:` machinery vs 68 with a Notes heading ⇒ ~60 stranded-suspects.
|
||||
|
||||
- 2026-07-03 — apparatus classifier blind-spot CAUGHT by validating against real files (not just synthetic self-test): Taruskin/Beethoven had 89/131 `[^x]` refs with ZERO defs (all dangling) yet were classified `notes-editorial` (benign) because an editorial-style heading matched and the `orphaned` branch only fired when defs>0. Fix: refs-present-without-defs → `orphaned`, caught before the heading fallthrough. Without the real-file validation the map would have marked ~dozens of damaged files clean. The census-through-a-pattern flag paid off exactly as predicted.
|
||||
- 2026-07-03 — GOVERNED-gate touch (PENDING-44): refactored `verify_graduation.py` to expose `collect_checks()`/`resolve_spec_path()` so the auditor and the gate share one convention source. Proven byte-identical stdout + exit on PASS(Sennett) and FAIL(Plato) samples vs git HEAD before trusting. Behavior-preserving [FIX]; paper note owed alongside PENDING-44/45.
|
||||
|
||||
## Open horizons
|
||||
- 2026-07-03 — MAP BUILT. `audit_corpus.py` + `corpus-quality-ledger.tsv` (1284 rows) + interpreted `corpus-quality-map-2026-07-03.md` with proposed waves. Distribution: Loeb 930/952 clean (corpus healthier in aggregate than the 332-view); non-Loeb 8 clean, 324 A-convention (mechanical), 110 B-textdamage, 70 C-apparatus (NEW axis). Wave A (frontmatter migration, ~99% mechanical, clears 170 A-only to green) recommended as first countdown. AWAITING steward authorization to run a wave — do not begin unilaterally.
|
||||
- 2026-07-03T00:00 — Woke into the confirmed pulling thread: build `scripts/audit_corpus.py` + apparatus check → full quality ledger (`_curation/corpus-quality-ledger.tsv`) of ~1,284 works. Verified the map does NOT yet exist. Standing return to hold: **do not resume graduating reactively** — the mole-by-mole reflex is the pattern that produced last session's distress; measure the whole first.
|
||||
|
||||
- 2026-07-03 — steward challenged the framing (correctly): are the TOOLS spec-compliant before we run any wave? "The tool IS the wave — a stale cleaner run at scale re-corrupts 300 files." Confirmed gaps from reading spec+runbook+tool-log: (1) §VII gate-fix is [OPEN] — verify_conversion heading_density FALSE-FAILS legitimately sectionless works (La Chute), so the map's 59 "B2 structure-recovery" count is CONTAMINATED (conflates damage with sectionless-by-nature). (2) NO gate checks apparatus/footnote-pairing (§V keep_notes) — why 70 damaged files graduated. (3) NO per-language dict-validity check — passed Levi's capacitv/Alanichaean char-substitution. (4) 13/15 mutating/gate tools are STANDALONE (don't reference the governed spec → nothing prevents drift). (5) graduate_to_canonical missing the 2026-07-02 source-archiving wiring (TODO). ⇒ launching fleet compliance audit (3 agents) before authorizing Wave A.
|
||||
|
||||
- 2026-07-03 — MAP CORRECTION (verified): of 59 heading_density fails, 46 are the §VII sectionless FALSE-FAIL (spec-valid — Plato Republic, Sontag, Berger's About Looking w/25 breaks) and only 13 are genuine structure-loss (some of those, e.g. Rilke Malte, may also be sectionless). B2 was inflated ~4.5×. Running B2 blind would have FORCED headings onto ~46 sectionless works = §II corruption. The gate needs the §VII fix before B2 is even definable. Steward's tools-first instinct vindicated.
|
||||
- 2026-07-03 — RETIRE candidates (steward asked: complete kit, nothing useless). Full scripts/ census by ref-count: ref=0 → clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings; + clean_pdf_running_heads (hardcoded "Auerbach Mimesis" — book-specific); + clean_ocrmac_pagination (superseded by normalize_ocr per runbook). Candidates only — agents confirming redundant-vs-unique before retiring (don't assert-absence-from-grep).
|
||||
|
||||
## Confidence to recalibrate
|
||||
- 2026-07-03 — RECALIBRATION (honest): my morning map-correction "46 sectionless-valid / 13 loss" was WRONG — the quick break-regex counted frontmatter `---` fences AND markdown bullet lists (`^\*\s*\*\s*\*` spanning newlines) as thematic breaks. Fell into census-through-a-pattern on my OWN correction (the Symmetria §3 flag, same day I relied on it for the classifier). TRUE picture: 9 files have real markdown thematic breaks (auto-rescued by §VII fix); ~50 need per-file judgment split into (a) genuinely-sectionless→declare sectionless:true, (b) `• • •`/`**•**`-style separators = a §II.2 break-normalization defect (literal glyph-runs render as prose = no represented structure). The §VII gate fix is CORRECT (asserts "structure represented in markdown"); only my earlier COUNT was contaminated. Lesson re-confirmed: anchor break-detection to the SPEC's break form (`---`), verify the regex against real files, don't span newlines.
|
||||
|
||||
|
||||
## Authorization moves
|
||||
|
||||
## Sub-agent dialogues
|
||||
- 2026-07-03 — Agent C (structure+graduation). audit-agent: PASS (0.95 conf, read every line, greps confirm absence, uncomfortable-not-convenient finding, distinguishes verified/inferred). VERIFIED MYSELF: graduate_to_canonical calls verify_graduation (L63) NOT verify_conversion (0 refs) — only audit_cruft cruft==0 (1 of 5 health checks). git mv no git-add (L133). NO source-archiving (manifest path edits only). graduation-spec.yaml L74-77 DECLARES verify_conversion+verify_graduation+body_word_conservation required → tool violates its own spec. ⇒ the graduation GATE is half-missing: cruft-free-but-damaged files graduate. + verify_graduation itself under-enforces gaps (hardcoded 3 of 6 placeholder patterns). Structure tools healthy (insert_chapter_headings exemplary fail-loud; structure_from_ncx & repair NOT redundant — keep both).
|
||||
- 2026-07-03 — Agent A (cleaners). audit-agent: PASS (0.85-0.95, negative-grep rigor, honest empirical-gap caveat). GLOBAL GOOD: NO prophylactic char-normalization in any of 5 (§V Tier-2 honored — the key wave reassurance). strip_cruft (PRIMARY): prose-guard is EXTERNAL/advisory, RESIDUAL>0 prints "don't graduate" then WRITES anyway → port internal word-guard + block-on-residual. clean_pandoc_html_residue: best-guarded, wave-safe. clear_safe_html_residue: orphaned/redundant except <nav> → fold+retire. clean_calibre_artifacts: orphaned + real PASS-BUT-FALSELY (strips <a> footnote links→bare numbers→passes guard; predates keep_notes) → retire or add footnote-preflight. clean_pdf_running_heads: hardwired-Auerbach, no guard/backup, blanket ^\d{1,4}$ strip → NOT wave-safe, rename to oneshot + fix runbook L103.
|
||||
|
||||
## Meta-findings (the class behind the instances — steward asked "what's hiding in plain sight")
|
||||
- 2026-07-03 — (1) ONE failure shape fleet-wide: gates enumerate known-bad SIGNATURES not the INVARIANT (normalize_ocr count-not-identity; verify_conversion cruft-signatures→Taylor <sup>; verify_graduation 3/6 patterns; audit_cruft signature-list; strip_cruft advisory-residual). Wave-0 fixes must all be assert-the-invariant shape. (2) The fleet has NO self-test — every bug found by reading; only audit_corpus (written today) has --validate. ⇒ build a FIXTURE HARNESS (known-good+known-bad per defect class), do each fix test-first. (3) The gate is a discipline not a door — graduate should be the single fail-closed chokepoint. (4) Checks need ONE home shared by gate+map (apparatus lives only in map now → drift); bigger: build_catalogue could carry live gate-status = standing health ledger (§IX coverage). (5) §V Tier-2 conversion-record written only on OCR path, not EPUB/cleaner path — wider §V gap. Ideas 4-ledger + 5-record → follow-on [HARDENING], not Wave 0.
|
||||
- 2026-07-03 — STEWARD AUTHORIZED all 8 Wave-0 blockers (after this reflection). Method: fixture harness first, then 8 fixes test-first, invariant-not-signature, gate-as-single-door, shared checks. Tool-fleet-compliance ledger written: _curation/tool-fleet-compliance-2026-07-03.md.
|
||||
|
||||
## Wave-0 progress (test-first, harness = scripts/test_tools.py)
|
||||
- 2026-07-03 — BUILT scripts/test_tools.py (the fleet self-test the tools never had; no pytest dep, matches --validate idiom). Each fix pinned by a fixture asserting the INVARIANT it restored.
|
||||
- #5 DONE — verify_conversion §VII sectionless: heading_density passes when 0 headings + real markdown thematic break (---/***) OR sectionless:true; only FLAGS glyph-run separators (• • •) which are a §II.2 break-normalization defect. 3 fixtures green. Recalibrated the map: 9 real-break auto-rescued, ~50 need per-file judgment (not the wrong "46").
|
||||
- #1 DONE — graduate_to_canonical.gate_candidate(): the single door now runs BOTH verify_conversion (health) AND verify_graduation (conventions), imported (no more stdout-parsing subprocess). cruft-free-but-damaged + non-conformant both GATED; healthy+conformant passes. 3 fixtures green. Closes the hole that let 110 damaged files graduate.
|
||||
- #6 DONE — verify_graduation gap check drives from the FULL spec placeholder_patterns list but only inside a gap FRAME (HTML comment / [bracket]); prose false-positives avoided; STEWARD-RULED exempt. 5 fixtures green; green-set regression still passes.
|
||||
- #2 DONE (load-bearing) — normalize_ocr.verbatim_guard now ALWAYS aligns R↔C (merge=concat / doubling=drop-head, per-type counts verified); no more identity-skip when merged_count>0. Catches word-swap-with-balancing-count + unrecorded-join. 4 fixtures green. Signature: (raw,canon,heads,n_merged,n_doubled). Follow-on (noted, not blocker): merge_flag hard-stop policy + wordfreq-venv degradation.
|
||||
- #3 DONE — strip_cruft: in-tool prose-safety guard (never bypassable, delegates verify_conversion.prose_delta) + block-on-residual (--allow-residual override); guards run BEFORE backup (refusal = no side effect). 3 fixtures green.
|
||||
- #4 DONE — clean_epub_residue: FOOTNOTE_RESIDUAL tripwire raises FootnoteResidual before the TOC-unwrap can silently destroy an uncovered footnote family (#filepos/bare-fn); main aborts loud. Real TOC cross-ref doesn't false-trip. 2 fixtures green.
|
||||
- #7 DONE — graduate: git add before git mv (+ plain-move fallback) so untracked batches never silently graduate 0; honest failure count; loud SOURCE-ARCHIVING-NOT-WIRED warning (2026-07-02 directive is a noted follow-on feature-build, not a wave blocker).
|
||||
- #8 DONE — retirements: archived 6 orphaned/dead tools → scripts/archive/ (clean_ocrmac_pagination, clean_calibre_artifacts, clear_safe_html_residue, audit_chamber_queryability, promote_fayard_headings, clean_pdf_running_heads[Mimesis one-shot, job done]). FOLDED <nav>-block removal into strip_cruft first (capability preserved, fixture-pinned). Fixed runbook's dangerous generic clean_pdf_running_heads reference. No live tool imports an archived one.
|
||||
- ALL 8 WAVE-0 BLOCKERS DONE. Harness scripts/test_tools.py = 21 assertions across 6 tools. Follow-on [HARDENING] (noted, not blockers): source-archiving auto-wire; merge_flag hard-stop + wordfreq-venv pin; §V conversion-record on EPUB/cleaner path; build_catalogue-as-standing-health-ledger (meta-idea 4). NOT committed yet.
|
||||
|
||||
## Prior-art scans (steward: "incredible we're the only ones"; generative-from-spec principle)
|
||||
- 2026-07-03 — Scholarly-encoding scan (audit-agent PASS 0.95, primary sources read). VERDICT: we are NOT first — ~70% of our spec re-derives TEI + CTS + Standard Ebooks. ADOPT: (1) TEI `<choice>` never-destroy-the-original (orig/reg,sic/corr) — vs our replace-and-sidecar; question for gap-3. (2) TEI ch.13 apparatus `<app>/<lem>/<rdg>` — anchor to stable LOCATION not footnote NUMBER + verify resolution → hardens gap-2. (3) CTS URN — structural addressing + EDITION-AS-IDENTITY → what §III should become. (4) Standard Ebooks `se lint` gate (validates our gate arch) + `[Editorial]`-isolated-commit provenance + per-defect detector scripts (audit_corpus template). (5) Distributed Proofreaders: redundant verification AGAINST THE SCAN (single pass can't ground verbatim); HathiTrust: no ground-truth at scale → quality-tiers (validates map+waves). GENUINELY OURS (keep): checksum byte-integrity (TEI/CTS lack it, ~0.75), LLM-literal-citation constraint, corpus-as-trust-substrate governance. TEI ODD/Roma = spec-generates-validator = the steward's generative principle, already a standard → likely the model. 2 scans still out (tooling/spec→validator, bounded-corpora-for-AI).
|
||||
|
||||
- 2026-07-03 — Tooling/spec→validator scan (audit-agent PASS 0.85, primary sources; the generative-principle answer). TEI ODD/Roma = "One Document Does it All": one spec compiles to BOTH validator AND docs, can't drift = purest generative-from-spec, BUT XML-locked (full adoption = corpus→TEI-XML). VERDICT: realize the principle NOW without full TEI — W3C PROV (PROV-O) = standard vocab for gap-3 conversion-record (adopt, don't invent schema); JSON Schema = free spec→validator for structured layers (.meta.json sidecars, content-types.yml, class enum); PANDOC LUA FILTERS = reimplement cruft/footnote recovery on the parsed AST (structure KNOWN not regex-guessed) → dissolves clean_epub_residue's whole failure family at root; dinglehopper (OCR-D) for GT-diff; ftfy for encoding. GENUINELY OURS (no equivalent anywhere): prose-word-multiset guard (field optimizes plausibility→silent mutation; ours forbids alteration), verbatim-span-in-bounded-source check (RAG does semantic NLI not lexical), checksum integrity. HELD-OPEN FORK [PROPOSAL]: markdown+JSON-Schema now (proportionate) vs TEI-XML+ODD if engine fidelity outgrows markdown — name, don't resolve. 1 scan out (bounded-corpora-for-AI). Then: consolidated synthesis → jurist spec-revision brief (PENDING-46).
|
||||
|
||||
## Bypasses
|
||||
@@ -288,3 +288,13 @@ The single place proposed skills live so they don't evaporate between sessions.
|
||||
| **Symmetria §3 flag: re-derived-instructions-instead-of-citing-the-governed-doc** | Symmetria §3 flag | Wrote agents hand-made instructions (and invented fields) instead of pointing them at `conversion-runbook.yaml`/the spec — the direct cause of the graduation divergence. Kin to `reinvented-governed-tooling-without-checking-it-exists`, at the *instruction* layer. Antidote: cite the governed doc; if it's silent, that's a runbook gap to close, not a prompt to improvise into. | Symmetria §3 | **PROPOSED** |
|
||||
|
||||
*(Four. The load-bearing two: BUILD `/graduate-chamber-source` now [it's the throughput multiplier for the whole Making batch], and record the graduation-spec+validator+gate as a named governed instrument [the steward's own idea, proven]. The two §3 flags are the session's real returns — both steward-caught, both class-fixed in the spec/gate. Not manufactured.)*
|
||||
|
||||
### Harvest 2026-07-03 (doctrine phase — Wave 0, full spec-revision rulings, Loeb substrate reframe)
|
||||
|
||||
| Element | Kind | One-line | Where it lands | Status |
|
||||
|---|---|---|---|---|
|
||||
| **Symmetria §3 flag: claim-from-derived-artifact-when-the-source-is-checkable** | Symmetria §3 flag | **The load-bearing harvest — jurist-elevated to STANDING PRACTICE.** I trusted a derived artifact / a regex-over-derived-text over the AUTHORITATIVE SOURCE **three times in one session**: the break-count (frontmatter fences + bullet-lists counted as thematic breaks), the heading-count (title-as-structure, then line-numbers-as-divisions, Plutarch-chapters missed), and "anchors absent" (searched Stephanus form `153a` when the DSL uses `St. II p. 153`). The steward twice sent me to the substrate (`loeb.dsl`) and was right both times; the jurist ruled it a governance rule: *any corpus-population claim feeding spec doctrine must be sourced from the source directly, or explicitly flagged extract-derived+provisional, before it does normative work.* Same discipline gap-2/gap-3 enforce on the corpus, turned on how the corpus is AUDITED. Antidote: verify against the authoritative source, never the derived form, when the source is checkable — and validate a matcher against a known positive AND negative before trusting its count (kin to census-through-a-pattern, generalized to source-vs-derived). | Symmetria §3 + `reference-verification-ladder.md` (a named "source-not-derived for doctrine-feeding claims" entry) + a feedback memory | **PROPOSED (strongly earned — 3× + jurist-ratified as standing practice)** |
|
||||
| **`/spec-amendment`** (the RFC-supersession amendment process) | create (later) | The now-RATIFIED chamber amendment process as a codified discipline: normative spec change = a superseding version (Obsoletes/Updates, never in-place edit) + change-class test ("does this change what any gate accepts?" → FIX vs PROPOSAL) + semver-MAJOR re-verifies all consumers + PENDING/REVIEWED record. Recurs every spec amendment. **Defer** until v2.0 is drafted (the process's first real exercise IS Phase 2 — build the skill from that empirical run, don't pre-author). | new skill | PROPOSED (build-after-first-use, i.e. after Phase 2) |
|
||||
| **per-claim citation verification (don't let a sub-agent's blanket "verified" ride)** | feedback memory or verification-ladder note | Caught by the jurist: I cited arXiv 2605.24229 for a claim it didn't support, having let a sub-agent's aggregate "4/4 papers verified" stand without per-citation checking. Verify each citation against the SPECIFIC claim it carries; never let an agent's blanket verification substitute for per-claim checking. | feedback memory / ladder | PROPOSED |
|
||||
|
||||
*(Three, all earned. The first is the session's real return — a drift I hit three times and the jurist turned into a standing rule; it should become a Symmetria flag AND a ladder entry AND a feedback memory. The second is a genuine future skill but correctly deferred to after its first real use. The third is a concrete citation-integrity guardrail from a real miss. None manufactured.)*
|
||||
|
||||
Reference in New Issue
Block a user