David/dotfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

session 2026-08-06: Constraint #1 wording (steward-enacted) + PENDING-106/107 corrections + PENDING-89 docket; BMF stopped and disabled; chavruta retrieval measured

Browse Source
This commit is contained in:
David F Glidden
2026-08-06 16:56:17 +02:00
parent fb4e171d9e
commit 3b0730d593
9 changed files with 283 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
claude/memory/project-L1-reliability.md
+19 -2
View File
@@ -8,7 +8,7 @@ metadata:
node_type: memory
type: project
originSessionId: fd7dd184-f64d-4f13-a1bd-abe1fa26192e
modified: 2026-08-04T08:01:19.473Z
modified: 2026-08-06T14:49:08.264Z
permalink: claude-memory/project-l1-reliability
---
@@ -42,7 +42,10 @@ permalink: claude-memory/project-l1-reliability
- **B1.1 fan-out cap is IMPLEMENTED AND WORKING — the code is not the problem.** `causal_edges_per_event_cap: 20` (`temporal/types.ts:201`), applied at `pipeline.ts:276`. Edges created **today: max in-degree exactly 20, avg 13.4, zero violations.** Edges created **pre-2026-06-23: max 629, avg 67.6, 4,677 nodes over cap.**
- **Therefore the defect is DATA, not CODE.** 836,467 edges / 813,178 chains were minted 2026-04-18 → 2026-06-23 under ungoverned fan-out. The fix landed *after* the damage, and the legacy graph taxes every future operation because `getChainsContainingSeq` scans all chains regardless of how disciplined new writes are. **This is why no ingest ever completed: cost per event grew with corpus size — divergent, not merely slow.** It also explains #65's "each attempt fails differently" — same wall, different points on the curve.
- **`coherence_evaluated = 0` of 813,178 chains — 0.00%, not one, ever.** June recorded "97%+ unevaluated"; it is now categorically zero. The epistemic layer that would prune chains has never executed. Separate from the pin; unfixed by the repair (a rebuild resets the count, it does not start the evaluator).
- **REPAIR RUN 2026-08-03 (steward-authorized).** `cm-ctl.sh repair --yes` — wiped `sqlite/ lance/ reasonchain/ perception/`, **preserved `logchain/` (145 MB, 109 files)**. Backup `mindfabric-00-20260803-wedged-preinvestigation` (1.4 G, 256/256 files, `quick_check: ok` ×4) taken **before** any change. Replay in flight, fan-out governed from the first event (`max_in=2, over_cap20=0`), **~10 nodes/min → ~33 h** for ~19,925 events. Finite now where it was divergent. Survives session end (launchd `com.capablemind.bettermemories`). **⚠ No `KeepAlive`** — a crash leaves it down silently. Steward has caffeine on indefinitely.
- **REPAIR RUN 2026-08-03 (steward-authorized).** `cm-ctl.sh repair --yes` — wiped `sqlite/ lance/ reasonchain/ perception/`, **preserved `logchain/` (145 MB, 109 files)**. Backup `mindfabric-00-20260803-wedged-preinvestigation` (1.4 G, 256/256 files, `quick_check: ok` ×4) taken **before** any change. Replay in flight, fan-out governed from the first event (`max_in=2, over_cap20=0`), **~10 nodes/min → ~33 h** for ~19,925 events. Finite now where it was divergent. Survives session end (launchd `com.capablemind.bettermemories`). ~~**⚠ No `KeepAlive`** — a crash leaves it down silently.~~ Steward has caffeine on indefinitely.
- ⚠ **THAT CLAUSE WAS FALSE WHEN WRITTEN, AND INVERTED — struck 2026-08-06.** The plist has carried `KeepAlive { SuccessfulExit: false }` **and** `RunAtLoad: true` since **2026-03-07** (mtime checked; the file has never been modified). So the truth is the opposite of what was recorded: **a crash — any unsuccessful exit, including a `kill` — RESTARTS it. A *clean* stop is the only thing that leaves it down.**
- **This is not a bookkeeping error; the false note is why the 08-04 decision silently failed.** On 08-04 the steward authorized killing the replay and the session recorded *"BMF is **down and staying down**"*. A kill is an unsuccessful exit → `KeepAlive` restarted it. **PID 1308, started Tue 2026-08-04 20:23:28, ran 1 d 20 h.** Neither party noticed for two days, and on 2026-08-06 the executor repeated "BMF is down and staying down" from this tracker **without checking the substrate**. The record asserted the absence of the exact mechanism that reversed the decision, and that assertion is what stopped anyone re-checking.
- **Instance of INC-2026-07-28-01's surviving finding (2), at our own scale, and stronger than PENDING-107's:** Constraint #1 was a control that was never tested. This one *was* tested, in the ordinary course of work, and failed silently while we reasoned from the false state. Transferable form: **a decision to stop something must be written where the restart mechanism can read it, or it is a preference, not a decision.** State was changed; policy never was. Maps onto ADR-014 constitutional bounds — runtime parameter vs. the bound that governs it.
- **Other findings, unfixed:** **4,648 empty Lance index directories** under `vector_chunks.lance/_indices/` (4,653 dirs, 5 real), leaking ~40/day since April. **#176 confirmed in source** — `BackupOrchestrator.countUnprotectedEntries` returns `0` when `lastBackupSuccess` is null, i.e. never-backed-up reads as fully protected.
- **GH backlog swept** per Seb's own standing instructions (`#170`, filed 2026-04-23, **never delivered** — circle-forward push failed, GitHub fallback unread for 3½ months). Closed `#125` (superseded: vector storage moved to LanceDB) and `#151` (done by `4f5b870`); narrowed `#121` to its live second cause (latency baseline frozen after first N samples, `resource-monitor.ts:145`).
- **Pattern across the day:** governors exist and do not engage — coherence evaluation never run, `ANALYZE` never run, `countUnprotectedEntries` safe-cases an absence, drift-check 3-of-5 families inert, verification ladder 71-of-75 entries never cited.
@@ -201,6 +204,20 @@ Each verified against `BetterMemories.io@3bc8b75` source. Re-verification agains
## Chronological log (most recent first; append substantive moves)
### 2026-08-06 — the 08-04 decision made to hold, and three write-path findings while making it
*(⚠ note in passing: **2026-08-04 has no entry in this log** — the session that produced PENDING-94 updated "Current state" but never appended here. Debt, not repaired in this entry.)*
**The stop, this time verified.** `launchctl bootout` + `launchctl disable` on **both** agents — `com.capablemind.bettermemories` (loaded, pid 1308) and `com.capablemind.bmf` (unloaded, but carrying **unconditional `KeepAlive: true`**; it would have fought for port 3011 if it ever loaded). `disable` persists in launchd's database across reboot, closing the `RunAtLoad` door that the 08-04 kill left open. Plists **backed up, not modified** (`*.plist.bak-20260806-164720`). Verified after: **port 3011 closed · no capablemind agents loaded.** Revert: `bmf-stop-and-keep-stopped.sh --revert`.
**The prompt tax, and it was never the backlog.** `UserPromptSubmit hook timed out after 5s — output discarded` had been recurring for the steward. Cause: **`/v1/observe` takes longer than the hook's own `BM_HOOK_TIMEOUT` (3000 ms)**, so every observation aborts, queues, and the *next* invocation tries to drain it and aborts again — self-sustaining, and **independent of queue depth**. Proven by counterfactual: parking a 10,485-item backlog changed hook latency **not at all** (3.11 s → 3.11 s; the queue refilled from live `PostToolUse` traffic within minutes). The executor's backlog-depth diagnosis was **wrong and was corrected by measuring**. Resolved by the stop: **3.11 s → 0.22 s** — `fetch` to a closed port fails immediately rather than aborting at the timeout. That was written down as a prediction first and tested by the script, not asserted after.
**Three write-path findings for Seb — measured, not inferred:**
1. **`relationships=31556 ms` on TWO entities**, against a 30 s pipeline budget → `Deferred (cursor held)`. **140 pipeline timeouts / 70 cursor-held defers per 2,000 stderr lines.** PENDING-93's densification made concrete at the write path, and the mechanism starving ingestion.
2. **`/v1/recall` returns HTTP 200 in 9.32 s** — 3× the hook's recall timeout, so hook-initiated recall can never succeed. Sits directly beside this tracker's `retrieval_count = 0` across the April–June graph. **Ingestion-never-completes and recall-never-worked, seen from a third end.**
3. **`/v1/observe` is effectively serialized server-side and collapses under concurrency** (measured on the real queue): conc 1 → 2.00/s, 100% · conc 4 → 0.47/s, 85% · conc 12 → 0.15/s, 12% (21 timeouts of 24). Parallel writers are **13× worse**, not faster.
**Artifacts.** Out-of-band drainer at `~/dotfiles/scripts/drain-hook-queue.mjs` — unlinks only on `res.ok`, halts after 10 consecutive failures, resumable, `--dry-run`; the concurrency measurements are written above its default so it is not "optimised" back. It **self-stopped on its first full run** (50 drained, 15 failed, nothing lost) *because of* finding (1), which is how finding (1) surfaced. **10,485 observations preserved, NOT deleted**, at `~/.capablemind/hook-queue-parked-2026-08-06`. ⚠ **Do not resume draining until the entity pipeline is healthy** — pushing 10 k observations in adds entities → relationships → more of the very densification that is timing out, while L1 is blocked on Seb.
### 2026-08-03 — The wedge named, ANALYZE applied, rebuild in flight; the code was right and the data was not
- **Entered L1 against the standing baton rule, on explicit steward lift.** The session had first proposed a *clasp* build (M4 Pro as circle-inference clasp per 43L/43M) to unblock ingest — measured and abandoned: §4.13's 2000 ms budget vs 192 ms transatlantic RTT, and a benchmark of Qwen3.6-35B-A3B-8bit on the M4 (median 2927 ms for 150 tokens, ~92-token break-even). **The clasp was the wrong lever** — the instance was never inference-bound. Profiling the running process answered in five minutes what reasoning from specs had not.
- **Method that worked, and is the transferable lesson:** ask the machine, not the specs. `sample` → main thread pinned in a timer callback; SIGUSR1 → CDP inspector → exact JS frames; read-only row counts and `EXPLAIN QUERY PLAN` off a *backup copy*, never the live DB.