David F GliddenandClaude Opus 5
ef6fa94dcd
The record is updated where information ARRIVES and never where a reader LOOKS.
Three instances found the same day, each with the correct information already
written down elsewhere:
REVIEWED-81 (AUTHORIZED 07-28) -> PENDING-81 still reads open
the ARC open-work register, which -> those 6 trackers still read live
says in its own description that
it supersedes them, with a
per-file verified verdict
register entry A2: "the content -> register entry B3, one page below:
half (B3) is now unblocked + "[OPEN - gated on A2]"
started" (06-17)
The third is the sharpest: both halves sit in the SAME document, so no tooling
gap explains it. ~/CLAUDE.md already rules this under Memory Discipline
("mark the superseded record as superseded"); the rule is being applied to facts
and not to trackers, PENDING items or register entries, which are also facts
about state.
Executed under steward authorization, arrears only:
- 16 trackers stamped `superseded_by:` + `superseded_on:` AND a visible in-body
banner. The banner is the point — metadata alone repeats the defect, since the
finding is about what a reader sees. 9 obsoleted by the 2026-07-07 MemPalace
wind-down; 7 already carrying the register's own verified `DONE-retire`.
- B3's label corrected, with the two-month error left visible rather than
silently amended.
- Nothing moved, renamed or deleted: 381 pointers resolve, 0 dead, 0 mis-authored.
HARVESTED BEFORE STAMPING, and this is why the harvest came first:
`project-mempalace-library-incremental-remine-strategy` carried a steward-verbatim
rule that is instrument-independent — never run a single multi-day index; order by
criticality; gate each tranche on READBACK, not on write-success (933,576 drawers
once stored successfully and could not be read back at all). It lived only inside
a MemPalace tracker, so retiring the instrument would have retired the rule with
it. ~/CLAUDE.md names that exact hazard: "state the obligation first and the
instrument second, or the next retired tool takes a rule down with it." Now
`feedback-bulk-indexing-runs-incrementally-with-readback.md`, and it applies to
studium-engine corpus work.
Held back deliberately: 2 correctly-parked files with named dependencies, 2 named
nowhere (returned to the steward), and 39 trackers unread — recorded as a gap,
not as a finding.
Measurement note: every earlier estimate of tracker staleness was wrong, because
both mtime and git-last-commit were reset across 283 files by the 2026-06-06
normalization sweep. The honest figure excludes it: median 90 days.
Filed as PENDING-142 ADDENDUM 2 per steward direction (fold, do not file anew).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Y6t6qx7cpaCu5xGdD36u4
2.9 KiB
name, description, metadata
| name | description | metadata | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| feedback-bulk-indexing-runs-incrementally-with-readback | Never run a single multi-day indexing/mining job. Go in tranches, ordered by criticality, and prove each tranche by reading back what was written before starting the next. Steward rule, 2026-05-04, after three days of compute produced unreadable output. |
|
Never run a bulk index/mine/embed job as one long pass. Tranches, ordered by criticality, with a write and readback check after each before the next begins.
Steward, verbatim, 2026-05-04:
"go with path A. then we make a choice on how to remine the library incrementally beginning with the most critical texts. I'm not losing days to this again."
Why: a failure only visible after days of compute is unacceptable, and it is the
normal outcome rather than the unlucky one. The 2026-05-01 → 05-03 chamber-library run
spent three days of CPU/MPS and wrote 933,576 drawers that stored successfully and
could not be read back at all — storage reported success, search returned Internal error: Error finding id. A run that writes without ever reading back cannot tell
"working" from "producing garbage at scale", so its own progress reporting is
uninformative. An earlier attempt burned nine hours to reach 30 of 371 files.
How to apply:
- Tranche it. No single job whose failure is invisible until it ends. Size the tranche so a wasted one costs an hour, not a weekend.
- Order by criticality. Index what is actually load-bearing for live work first, so the useful half exists even if the run is abandoned midway.
- Readback is the gate, not the write. Storage success is not the property you want; retrieval of what you stored is. Query each tranche before starting the next — this is the reference-verification-ladder's discrimination rule applied to indexing: a check that only exercises the write path cannot see a broken read path.
- Watch for a global setting applied to a heterogeneous corpus. The same run was compounded by one embedding model being forced across all collections; dimensions that suit a multilingual scholarly corpus need not suit anything else.
⚠ This rule is instrument-independent and outlived the tool that earned it. It was
learned on MemPalace, which was wound down 2026-07-07 (project-mempalace-winddown) —
and it was carried only inside a MemPalace tracker, so retiring the instrument would
have retired the rule with it. ~/CLAUDE.md names that hazard directly: "state the
obligation first and the instrument second, or the next retired tool takes a rule down
with it." Harvested here 2026-08-17, on the day the superseded tracker was stamped, for
exactly that reason. It applies now to the studium engine's corpus mining — the same
shape of work, a different tool.