dotfiles/claude/memory/session-2026-04-09-l1-audit.md

name, description, type, originSessionId

name	description	type	originSessionId
Session 2026-04-09/10 evening — L1 full pipeline audit, systemic fixes roadmap	Comprehensive L1 audit (4 parallel agents), 8 issues filed, Effect-TS adoption proposed as substrate. Research folder established. Full diagnostic-to-engineering-roadmap for Seb.	project	662625c1-73a8-4f3d-a7f1-4ed16a33864c

What we did

Research folder established

Created CapableMind-AI/docs/thinking/David/research/ for external project studies. Updated routing table in thinking/David/CLAUDE.md. Memory saved for future sessions.

External landscape research

• GitHub trending survey — evaluated 10 trending projects for CapableMind relevance
• Hermes Agent study (2026-04-09-hermes-agent-study.md) — trajectory-to-training pipeline, skill architecture (conditional activation), memory manager (provider plugin with single-provider constraint). Key insight: trajectory capture as training byproduct.
• MemPalace study (2026-04-08-mempalace-study.md) — operational assessment of our installed tool. 96.6% recall with vector-only search validates BMF's logchain philosophy. No governance layer, no developmental model — BMF goes beyond it. KG is empty (0 entities).

L1 recall path audit

• Recall path audit (2026-04-09-l1-recall-path-audit.md) — full code trace of query entry → results. Corrected 3 stale assumptions from PENDING-5: query router already has intent-based fast paths, classification confidence already threaded through (Amendment 61), stub embeddings already blocked (#15 fix). Real problems: Ollama query SPOF, BM25 broken in SurrealDB, normalization aggressive.
• Filed #124 (Ollama silent recall failure), #125 (BM25 broken), #126 (normalization)

L1 pipeline landscape

• Pipeline landscape (2026-04-09-l1-pipeline-landscape.md) — surveyed tools for each problem category:
  • LanceDB recommended for vector storage (embedded Rust/Node, Tantivy FTS, no compaction stalls)
  • @huggingface/transformers recommended for in-process embedding (eliminates Ollama SPOF)
  • p-queue recommended for dispatch backpressure (later superseded by Effect structured concurrency)
  • Graphiti (temporal fact invalidation pattern), Mem0 (conflict resolution), XState (state machines) noted as study-only

Full pipeline audit (4 parallel agents)

• Full audit (2026-04-09-l1-full-pipeline-audit.md) — most significant work of the session. Four agents read the entire codebase in parallel:
  1. March findings verification: 13/27 fixed, 5 partially fixed, 9 improved with logging. Fire-and-forget 22→19.
  2. Write path: Logchain solid, classification never drops events, module isolation works. HIGH: vector battery skip loses events permanently, entity module swallows all pipeline errors.
  3. Replay/bootstrap/infra: CRITICAL: modules report ready during replay (setReadyPartial/setReady are dead code, coverage annotations never wired). Frozen latency baseline still not fixed. Bug D (idle stall) IS fixed via activity lease.
  4. Module internals: Temporal query errors silently return empty. Training graduation still fire-and-forget. ~75 bare catch blocks, ~15 architecturally significant. In-memory state loss universal (anomaly: 14-day gap after restart).
• Filed #133 (CRITICAL: readiness lie), #134 (entity error swallowing), #135 (temporal query silent failure), #136 (vector battery skip)

Systemic fixes roadmap

• Systemic fixes (2026-04-10-l1-systemic-fixes.md) — THE document for Seb. Maps every finding to a systemic solution:
  • Effect-TS as substrate: typed errors, structured concurrency, Layers for DI, supervised fibers, built-in tracing. Adopted because team is scaling — do it once correctly. Strangler-fig adoption over 4 weeks.
  • Position-based readiness: derived from stream position, not asserted. No new deps.
  • better-sqlite3 snapshots: module state survives restart. Already in dep tree.
  • LanceDB + transformers.js: decouple vector from SurrealDB, embedding from Ollama.

What we decided — and why

1. Research folder in thinking/David/research/ — David is doing increasing landscape scanning. Needs a permanent discoverable home, not ephemeral session notes.

2. Effect-TS over neverthrow — Initially recommended neverthrow (simpler, solves only error handling). David pushed back: "We are a two person team TODAY, but we are ramping this up to scale." The prime directive demands doing it once. Effect solves errors + concurrency + DI + tracing + resource lifecycle in one framework. neverthrow would mean adopting a second tool later. Effect is the right long-term bet despite higher upfront cost.

3. LanceDB over sqlite-vec — LanceDB has built-in Tantivy FTS (real BM25), Arrow-based columnar storage (no compaction), native Node.js bindings. sqlite-vec is simpler but caps at ~100K vectors and needs separate FTS5. Given scaling intent, LanceDB is proportionate.

4. Full audit before proposing fixes — David asked "what's your confidence?" on the recall diagnosis. I admitted it was based on stale PENDING-5 data. David insisted on thorough audit first. The audit corrected multiple stale assumptions and found the CRITICAL readiness lie that wasn't visible from the recall path alone.

What's unresolved

• None of the 8 filed issues have responses from Seb yet — all filed today, Seb is active but busy with other priorities (Amendment 62 / BetterBridge)
• DN-RET-02 practitioner questions — 5 questions about score memorization under performance conditions, still unanswered by steward
• Effect-TS adoption needs Seb's buy-in — this is a foundational change to HIS codebase. The document makes the case but the conversation hasn't happened.
• MemPalace KG empty (0 entities, 0 triples) — decision pending on whether to populate
• ARC + chamber-library pushes still blocked on Gitea
• External Auditor response pending

Next actions

1. Discuss systemic fixes with Seb — bring the consolidated document. Effect-TS adoption is the biggest conversation.
2. Respond to External Auditor — assessment pending, what enters record, whether HOLD triggered
3. Answer DN-RET-02 practitioner questions — score memorization, crowding, re-learning
4. Push ARC + chamber-library when Gitea is resolved
5. Continue /audit of vault inbox — 16 files remain

Commits this session

• 487503e — research folder, recall path audit, MemPalace + Hermes Agent studies
• d1d4b16 — same (rebased after pull)
• 3088868 — L1 pipeline landscape
• c35f621 — L1 full pipeline audit
• ce60b4c — L1 systemic fixes roadmap

Issues filed this session

#	Title	Severity
#124	Ollama silent recall failure	HIGH
#125	BM25 scoring broken	MEDIUM
#126	Normalization aggressiveness	LOW
#133	Module readiness lie	CRITICAL
#134	Entity error swallowing	HIGH
#135	Temporal query silent failure	HIGH
#136	Vector battery skip	HIGH

No Thistleweld observations this session