session 2026-08-04: PENDING-92/93/94 + the replay-never-resumed finding + l1-replay-sampler

Filed PENDING-92 [HARDENING] idle ladder (cool/deep unreachable, spec §9A.1
divergence), PENDING-93 [PROPOSAL] event_seqs normalisation, PENDING-94
[ESCALATE] the resume floor — minCursor pinned at 0 by two non-participating
modules, so 13/13 restarts rebuilt from seq 0 and the catch-up branch has
never executed. Recall never worked either (retrieval_count = 0 across the
whole April-June graph); same fact from the other end.

Adds scripts/l1-replay-sampler.py (external read-only sampler, four positive
controls, refuses to run blind). Note to Seb pushed separately as
CapableMind-AI@ad285df.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
This commit is contained in:
David F Glidden
2026-08-04 15:30:56 +02:00
co-authored by Claude Opus 5
parent ad2fb07698
commit 9c2a7c985c
8 changed files with 603 additions and 5 deletions
+48
View File
@@ -754,3 +754,51 @@ Measured against the register itself before acting. Four corrections; the item's
**Verification:** 33/33 quotations in the package mechanically contained against source, 9/9 positive controls absent, instrument verified. Prototype checks: zero-JS gate 0, cycle-end clamp holds at three horizons and floors on a negative trajectory, monotonic over 800 sampled points, validator rejects 6/6 malformed genomes by name.
**Files affected:** `docs/AldineXXI-Codex/drafts/vignette-phase-1a-JURIST-PACKAGE-2026-08-02.md` (new); `tools/vignette-proto/` (built, committed `946b88b`/`1d40d4d`/`095be00`). No spec text changed, nothing integrated.
**Awaiting:** Steward relay to the jurist, and a decision on Q5 before the ruling is sought.
## PENDING-92 — The idle ladder's bottom half is unreachable, and the work that lives there has never run
**Date:** 2026-08-04
**Tag:** [HARDENING]
**Summary:** `cool` and `deep` are unreachable on this deployment — the only `warm → cool` path is `onAgentDisconnect()`, whose sole caller is the shutdown sequence — so deferrable `idle_only` work (background training, consolidation, Ollama batch windows) has never drained once.
**Rationale:** Sixth instance of the class named 2026-08-03 (governor exists and never engages), and the first found by looking for it rather than tripping over it. An inert control reports success: `bm_idle_state` has only ever reported 0 or 1, which reads as a healthy machine rather than a ladder missing its bottom half.
**Evidence (substrate, positive-controlled):** across the full `bmf.stderr.log`, 2,393 idle transitions in exactly two shapes — `active → warm` (1,197) and `warm → active` (1,196). **Zero** to `cool` or `deep`, ever. **Zero** `drained N deferrable idle_only work items` lines, ever. The positive control is the 2,393 itself: the grep demonstrably sees transitions when they exist, so the absence is measured, not assumed.
**Code:** `src/inference/idle-state-machine.ts:204` — *"warm and deep have no time-based exits from tick"*; `tick()` handles only `active` and `cool`. The single `onAgentDisconnect()` caller is `src/bootstrap.ts:2460`, inside the shutdown path (*"1. Stop MCP server and signal agent disconnect (44G)"*). `cool` is therefore reachable only while the process is dying, and `deep` — which requires 4h resident in `cool` — is unreachable by construction.
**Spec (the divergence):** `docs/specs/operations/local-inference-spec.md` §9A.1 (v1.6, Amendment 44G), HTTP-only/service-mode table: *"State machine transitions are identical — only the detection inputs change. 'Agent connected' maps to 'recent HTTP activity.' 'Agent disconnected' maps to 'no HTTP activity for cool idle threshold.'"* The spec requires a time-based path into `cool`; the code implements only literal MCP disconnect. The same section anticipates precisely this deployment: *"For idle to work beyond the session, the architecture needs launchd/systemd as the primary process with stdio MCP connecting TO the service."* That is mindfabric-00 exactly.
**Why [HARDENING] and not [FIX], though the divergence is scoped:** the *diagnosis* is FIX-shaped — code diverges from spec, and per the repo's methodology the spec wins. The *remedy* is not: enabling the descent switches on a subsystem that has never executed in production on any instance. Idle training cycles, memory consolidation and Ollama batch windows would run for the first time, on the steward's primary instance, against a graph mid-rebuild. Calling that "restoring intended behaviour" would be a self-assessment I cannot honestly make.
**Options:** (a) implement the spec's HTTP-activity-timeout path behind an env flag defaulting **off**, so the first descent is deliberate and observed; (b) implement unflagged, matching spec directly; (c) amend the spec instead — declare `cool`/`deep` out of scope for MCP-attached deployments and delete the unreachable states, choosing honest degradation over dormant capability; (d) defer entirely until the replay completes.
**Recommendation:** (a) sequenced behind (d) — build now on a branch, enable only after the replay completes and with `l1-replay-sampler.py` running, so the first-ever `warm → cool → deep` descent is watched rather than discovered afterwards. (c) stays live as the honest alternative if the idle subsystem turns out to be unwanted on laptops; what should not persist is a four-state ladder advertising two states it cannot enter.
**Files affected:** `src/inference/idle-state-machine.ts`, `src/bootstrap.ts` (BetterMemories.io). Spec unchanged — the code moves to the spec, not the reverse.
**Awaiting:** Steward authorization; then Seb review via PR per Constitutional Constraint #3 (no direct push to main on L1).
## PENDING-93 — `getChainsContainingSeq`: the rebuild buys a constant factor, not a complexity class
**Date:** 2026-08-04
**Tag:** [PROPOSAL]
**Summary:** `SELECT * FROM causal_chain WHERE EXISTS (SELECT 1 FROM json_each(event_seqs) WHERE value = ?)` is a full scan of every chain, unindexable as written; the 2026-08-03 repair reduces how many chains it scans but leaves the cost linear in chain count.
**Rationale:** This is the second of the two hot paths named by the 2026-08-03 CDP profile (the first, `getCausalEdgesFromSqlite`, was resolved by `ANALYZE`). It is the one that remains, and the repair does not address it. Filing it so the schema question is docketed rather than living only in the workstream tracker — PENDING-11/12 sat open four and a half months precisely because they were recorded where nobody re-read them.
**Measured:** 4.0 s per call against 813,178 chains (2026-08-03, off the verified backup). The current rebuild at 22,300/39,089 events (57%) already carries **71,225 chains against 6,803 nodes — 10.47 chains per node**. A linear extrapolation to full replay gives ~125,000 chains, a ~6.5× reduction against the pre-governor 813k — but it is a **floor, not an estimate**: chains-per-node and edges-per-node are both still climbing (edges/node has gone 4.85 → 8.0 → 11.29 across this morning), so the true figure is higher. Either way the scan stays linear, and every future operation pays it.
**Options:** (a) normalise `event_seqs` into an indexed join table (`chain_event(chain_id, event_seq)`), turning the scan into an index seek; (b) maintain a materialised seq→chain map alongside the existing JSON column, leaving the schema additive; (c) cap chain creation at mint time, attacking the population rather than the query — noting the 2026-06-06 finding that a new 2-node chain is created per non-extension, which is what makes chains outgrow nodes 10:1.
**Recommendation:** (a) is the durable answer and (b) is the reversible one; (c) is complementary to either and is where the growth actually originates. This is a schema change on L1 core — Seb's call, not ours. What we can supply is the measurement, which the sampler now produces continuously.
**Files affected:** `src/modules/temporal/storage-sqlite.ts:657` (query), schema migration (new). None touched.
**Awaiting:** Seb, via the co-authored L1 channel; steward relay.
## PENDING-94 — The replay has never resumed, only restarted: two modules pin minCursor at 0 permanently
**Date:** 2026-08-04
**Tag:** [ESCALATE]
**Summary:** `minCursor` is the **minimum** cursor across all 11 modules; `structured` and `training` have never processed a single event and sit at 0 forever, so `minCursor` is permanently 0, every start is classified `rebuild`, and the replay re-reads the entire logchain from seq 0 — **13 of 13 restarts, zero catch-ups, ever.**
**Escalation grounds:** touches **cursor persistence**, named in `~/CLAUDE.md` as an unconditional `[ESCALATE]` surface. Do not proceed without steward authorization and Seb.
**Why this is different from a performance finding:** it does not explain why events are slow. It explains why the ingest can never **complete**, at any speed. Those are separable, and conflating them is how the last four months went.
**The chain, traced:**
1. `module_cursors` after a 15-hour run: `anomaly` 24,291 · `safety` 24,291 · `budget` 24,290 · `security` 24,281 · `temporal` 24,277 · `entity` 24,034 · `blob` 7,889 · `preference` 1,942 · `vector` 1,490 · **`structured` 0** · **`training` 0**.
2. `replay-coordinator.ts:300` — `minCursor = Math.min(...[...cursors.values()].map(c => c.cursorPosition))`. One module at 0 zeroes the whole computation.
3. `:340` — `this.replayIsRebuild = minCursor === 0`.
4. `:368` — `if (entry.seq <= minCursor)` skip. At 0, nothing is skipped: the full logchain replays.
5. `:319` — a restored snapshot would raise `minCursor` to `snapshotSeq`. **"Snapshot restored at seq" appears 0 times in the entire log**, against 13 for its sibling "Restored cursors from database" (positive control: the code path runs and the grep sees it). The escape hatch exists and has never fired.
**Why `structured`/`training` sit at 0:** **0 deferrals and 0 pipeline errors each** — they are not failing, they simply never receive an event they handle. Positive-controlled: the same grep finds **103,130** deferrals for `vector` and **3,723** for `entity`, so it demonstrably sees these lines when they exist. Their cursor is therefore legitimately 0 and will remain 0 for the life of the instance.
**Second, independent pin:** `vector` holds at 1,490 after **103,130** deferrals. `base.ts:134` holds the cursor on `DeferrableError` *by design* — "Do NOT advance cursor — event will be retried on next replay." Correct per-module; catastrophic when a single held cursor gates a global minimum.
**The design assumption that fails:** the comment at `:332–340` reads `minCursor === 0` as "no cursors and no restored snapshot… errs safe". It cannot distinguish *fresh install* from *nine modules at 24,000 and two that never participate*, and it errs into the most expensive behaviour available — permanently.
**What this predicts, and the record confirms:** completion requires **one uninterrupted pass over the whole logchain**. At the repo's own documented-healthy 17 events/min, 39,089 events is a **38-hour uninterrupted run**; at the observed 2.32/min it is **12 days**. Any crash, restart, upgrade or repair inside that window returns the system to zero. This is `#65`'s *"each attempt fails differently"* exactly: each attempt was a fresh full replay dying at a different point on the same curve. It also explains why every genuine rate fix (ANALYZE 6.4×, B1.1 cap, N6) improved throughput and changed nothing about completion.
**Options:** (a) compute `minCursor` over **participating** modules only — those that have ever processed an event — excluding permanent non-participants; (b) take periodic snapshots so `:319` raises the floor, which needs no change to the minimum logic and is the smallest reversible move; (c) persist a separate replay high-water mark independent of module cursors; (d) let a module declare itself non-participating at registration so it is excluded by construction rather than by heuristic.
**Recommendation:** (b) immediately as relief — snapshots are an existing, already-wired mechanism that has simply never been used — and (d) as the durable answer, because it makes participation explicit rather than inferred. (a) is tempting and I distrust it: "has ever processed an event" is itself a heuristic and would silently re-break the moment a module's first event arrives late. **All of it is Seb's call; cursor persistence is not a surface we touch.**
**Confidence:** ~0.85 that the mechanism is as traced; ~0.7 that it is the dominant reason no ingest has completed since April. Stated because four rate-hypotheses were proposed and refuted on 2026-08-04 alone — though those were arithmetic coincidences and this is a traced code path over persisted state, which is a different epistemic character.
**Falsifier, already run:** any restart with `min cursor > 0`, or any `kind: catch-up`. **13/13 restorations report `min cursor: 0`; 6/6 replay classifications report `rebuild`; catch-up has never executed.**
**Files affected:** `src/core/keystone/readiness/replay-coordinator.ts:300/319/340/368`, `src/modules/base.ts:134`. **None touched.**
**Awaiting:** Steward authorization before any change; then Seb, as L1 core.