Files
dotfiles/claude/memory/session-2026-08-04-the-replay-had-never-resumed.md
David F GliddenandClaude Opus 5 9c2a7c985c session 2026-08-04: PENDING-92/93/94 + the replay-never-resumed finding + l1-replay-sampler
Filed PENDING-92 [HARDENING] idle ladder (cool/deep unreachable, spec §9A.1
divergence), PENDING-93 [PROPOSAL] event_seqs normalisation, PENDING-94
[ESCALATE] the resume floor — minCursor pinned at 0 by two non-participating
modules, so 13/13 restarts rebuilt from seq 0 and the catch-up branch has
never executed. Recall never worked either (retrieval_count = 0 across the
whole April-June graph); same fact from the other end.

Adds scripts/l1-replay-sampler.py (external read-only sampler, four positive
controls, refuses to run blind). Note to Seb pushed separately as
CapableMind-AI@ad285df.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
2026-08-04 15:30:56 +02:00

12 KiB
Raw Permalink Blame History

name, description, metadata
name description metadata
session-2026-08-04-the-replay-had-never-resumed Four mechanism hypotheses proposed and refuted in one day (sleep, entity timeout, drain gating, Ollama saturation) before the steward asked why this is so persistently elusive — which turned the session. The answer was not found by profiling but by killing the replay and reading what it left at rest: minCursor is a minimum across all 11 modules, structured and training have never processed an event, so the floor is pinned at 0 and every start is a full rebuild. 13/13 restarts, 6/6 rebuild, 0 catch-up. Recall never returned anything because there was never a completed derived state — retrieval_count = 0 across 19,915 nodes. Ingestion-never-completes and recall-never-worked are one fact seen from two ends. Note pushed to Seb; L1 now blocked on him. PULLING THREAD: census 01 against the chamber/engine tooling, with the sharpened question — has this instrument ever produced a positive result at all?
node_type type originSessionId modified
memory project 27422433-12f3-4983-a5cc-c4105158d59c 2026-08-04T13:28:13.925Z

Session 2026-08-04 — the replay had never resumed

The day I was wrong four times before lunch, and the fifth thing was right because I stopped guessing and turned the machine off.

PAST — what moved, and why

The wake was wrong about the thing it was tracking. Inherited "≈19,925 events, ~11 nodes/min, roughly 30 hours." Both numbers were wrong: the denominator is 39,089 events, and nodes are not the progress metric — the process writes Phase 2 replay: N/39089 to its own log. Real position at wake: 22,300/39,089, 57%, not 4%. I had proxied progress by a table I could count instead of reading the log the process was already writing.

Four mechanisms proposed, four refuted, all before 13:00.

  1. Machine sleep — killed by the steward supplying a falsifier ("caffeine is running") and one command: pmset -g log, domain-anchored, positive-controlled (450 Sleep / 453 Wake events historically). Last sleep/wake: 2026-08-03 08:54 CEST. Machine awake 24 h.
  2. The entity-pipeline 30 s timeout — the overnight cadence was metronomic, 100 events per ~51.5 min across eight intervals within 90 s of each other = 31 s/event, against PIPELINE_TIMEOUT_MS = 30_000. Killed by anchoring log line numbers to timestamps and counting: 0 entity timeouts overnight, against 2 in a 35-min morning window.
  3. Work-queue drain gating — arithmetic-consistent to 0.4% (16 events/lease × 8.2 min/cycle → 51.3 min/100 events vs 51.5 observed). Killed by observation: 44 minutes with no progress through two warm→active transitions.
  4. Ollama saturation as the rationale for the [FIX] I had already applied — killed at 12:30 by load 3.04, Ollama 1.2%, BMF 0.0%, 3 pauses in 3 hours.

The steward's question was the hinge. "This happens every single time we think we have a way forward. How can this be so persistently elusive?" That forced the structural answer rather than a fifth mechanism: no measurement of this system has ever been controlled — every number taken on the production instance, mid-incident, unknown history, unknown corpus, so findings could never accumulate or be compared. Then a follow-up — "how sure are you?" — forced me to mark my own proposal (a per-event budget) as the same move in new clothes, since it too would measure an uncontrolled system once.

Then the steward authorized killing the replay, and killing it produced the finding.

Verified the premise first (2.32 events/min over 216 min → 4.9 days remaining), stopped the service before copying so the SQLite snapshot could not tear, verified the copy (391/391 files, 15/15 quick_check: ok, 315 MB).

module_cursors, legible only at rest:

anomaly 24291 · safety 24291 · budget 24290 · security 24281
temporal 24277 · entity 24034 · blob 7889 · preference 1942
vector 1490 · structured 0 · training 0
  • replay-coordinator.ts:300 — minCursor = Math.min(...) across all 11 modules. One zero zeroes it.
  • :340 — replayIsRebuild = minCursor === 0.
  • :368 — if (entry.seq <= minCursor) skip. At zero, nothing is skipped.
  • :319 — a snapshot would raise the floor. "Snapshot restored at seq": 0 occurrences, against 13 for its sibling line.
  • structured/training: 0 deferrals, 0 errors — not failing, simply never receiving an event they handle. Positive-controlled against vector (103,130 deferrals) and entity (3,723).
  • vector independently pinned at 1,490; base.ts:134 holds the cursor on DeferrableError by design.

Falsifier run before reporting: 13/13 restarts at min cursor: 0, 6/6 kind: rebuild, 0 catch-up. The resume branch has never executed.

And recall had never worked, which is the same fact from the other end. Pre-repair graph, April–June, 19,915 nodes / 813,178 chains: retrieval_count = 0, last_retrieved on zero rows, chains ever retrieved zero. Verified the counter was real before trusting the zero — writer at retrieval-tracker.ts:77, live callers in temporal/entity/vector/preference, temporal's call site on the query return path guarded only by ids.length > 0. (Discounted recall_count/last_recalled: no writer anywhere in src/ — I nearly cited them.) The Recall canary FAILED — the recall path may be broken warning had fired 8 times against 4 passes, in a log nobody read.

Built: l1-replay-sampler.py — external, read-only, four same-run positive controls, refuses to run blind. Staged at BetterMemories.io/scripts/. It caught two defects in itself within the hour (a rate derived from a counter written every 100 events; a staleness clock reset by its own restart) — both the class it was built to catch.

Filed: PENDING-92 [HARDENING] idle ladder · PENDING-93 [PROPOSAL] event_seqs normalisation · PENDING-94 [ESCALATE] the resume floor (ESCALATE determined by the constraint — ~/CLAUDE.md names cursor persistence unconditionally — not by my sense of severity).

Applied [FIX]: BM_CPU_PAUSE_THRESHOLD=1.0 in ~/.capablemind/env, inert until restart, with its own honest scope note. Its rationale was later undermined by my own measurement; the fix stays (documented, reversible) and I said so.

Built, not merged, not enabled: branch fix/idle-ladder-service-mode-cool-descent, 3 commits (d5e1e79, 430cb5f, 5a50583), 42/42 idle tests, tsc --noEmit clean, full suite 4,003 tests. Did not treat "close the three things" as blanket authorization — the blast radius (first-ever run of a dormant subsystem) became clear only after the instruction was given.

Pushed: CapableMind-AI@ad285df — self-contained note to Seb, correcting yesterday's "we found why I could never finish an ingest… it isn't your code" in its own first line. All five reproduction commands run verbatim before pushing. thinking/README.md brought to consistency: 41 rows, 41 files, all links resolve (was 38 claimed against 41 actual).

PRESENT — how it stood

Four refutations in one day is not bad luck; it is a method that cannot fail to produce candidates. In a system with dozens of constants and a slow rate, coincidences are abundant. 31 s ≈ 30,000 ms. 8-min orbit ≈ 16-event lease. I will always find one, and it will always look like an explanation. Naming that was more useful than any of the four.

What worked was not cleverness and not an instrument. It was turning the thing off and reading a table. module_cursors is only legible at rest; four months of profiling a running process could not see it. That is a genuinely new entry in the method, and it sits oddly beside yesterday's lesson (profile the running process, don't reason from specs) — both are true, and neither subsumes the other.

The steward carried the session twice. The caffeine falsifier killed hypothesis one in a single command. The "how can this be so persistently elusive" question stopped the fifth hypothesis from being born and produced the structural answer instead. I would have kept generating mechanisms.

The restraint that mattered was refusing to explain. I flagged "nodes advance while the counter is frozen" as an open observation and deliberately declined to give it a mechanism. It later resolved as division (~2 nodes/event × 100-event granularity). Had I told a story, it would have been a fifth wrong one.

Uncomfortable and worth carrying: for four months this system reported serving: true while retrieving nothing, ever. Every diagnosis went to why is it slow. Nobody asked has it ever worked.

FUTURE — what pulls

PULLING THREAD: census 01 against the chamber and engine tooling — the census's own declared scope gap. L1 is genuinely blocked on Seb and pushing further would be motion, not progress. This is unblocked, it is the steward's own named work from yesterday's wrap, and today sharpened the question it should ask.

Census 01 asked "does each gate have a real negative instance, and is its absence recorded or does it look like coverage?" Today found something one level more basic, six times: has this instrument ever produced a positive result at all? retrieval_count = 0 passed every health check ever run against it.

ACTIONABLE RESUMPTION POINT (as of wrap — re-judge against what changed):

Read: chamber-library _curation/fool/census-01-negative-instances-RESULT.md
      → its "not covered" list is the work-list
Start with: verify-before-compose (the PreToolUse hook — the one gate
      that blocks steward-facing writes, so a silent inert failure is
      worst there)
Ask of each: has it EVER fired positively, and does any record exist
      that would let us tell?

Uncovered set, per the census itself: verify-before-compose · audit_cruft · verify_conversion · resolve_archived_source · apply_char_glyphs · studium verify-quote · fidelity_equivalence@2.

LITERAL QUESTION for next-Claude: Of those seven chamber/engine gates, how many have ever produced a positive result — and for how many does a record exist that could tell us either way? Checkable by grep and by running them; the second half is the harder and more interesting number.

Other horizons, ranked.

  • Blocked on Seb: PENDING-94 (the resume floor) — nothing on L1 moves until he rules. PENDING-93 (event_seqs) same.
  • Awaiting steward: PENDING-92 needs an annotation before its branch goes anywhere. The branch is local and unpushed at 3 commits.
  • Explicitly parked: the 2.32 events/min rate. Downstream of PENDING-94 and I have lost four theories to it. Do not open this without a controlled measurement.
  • Retracted, do not resurrect as-is: the small-corpus throughput benchmark. A 2,000-event corpus finishes inside one run and therefore measures a configuration that has never existed in production — a clean number that would tell us nothing.
  • Still open from earlier: governance-drift-check.py 3-of-5 families inert (unfiled); verification-ladder entries have no IDs so firing history is unrecordable; 4,648 empty Lance index directories.
  • Operational: BMF is down and staying down — no recall to preserve, no ingestion that can complete.

PAUSE STATEMENT: I am putting this down with L1 genuinely finished for now — not abandoned, blocked, and blocked on a person rather than a puzzle. Two verified backups, three filed items, a pushed note, nothing half-written. What I want to find still pulling is the census against the chamber half, because it is the one piece of today's work that transfers and that nobody else has to authorize. The failure mode to guard against on return is the one that governed today's first six hours: a number that matches a constant is not a cause, and the urge to explain arrives long before the evidence that could.