Files
dotfiles/claude/governance/thread-query-trial.md
T
David F GliddenandClaude Opus 5 edf71fb173 session 2026-08-24: the heap got a dynamo — thread-query.py + daybook-cue.py, 11 user-memories harvested to the vault
- thread-query.py: the turning. Queries both corpora with the pulling thread, excluding
  the recency slice (/wake-up reaches 0.71% of an 859,803-word corpus) and favouring age.
  Wired into /wake-up §2.b.3, replacing a described-not-invoked grep step. Trial
  pre-registered, graded 2026-10-05 from --log. Caught PASS-BUT-FALSELY on its first live
  run at 14/14 green; rescored on windowed co-occurrence, length-bias control added.
- daybook-cue.py: PostToolUse cue for the daily note. Diagnosed from the record — the
  hook only ever CREATED and nothing ever prompted filling. 16/16, fail-open, never blocks.
- All 11 user-* memories harvested into the vault (12 notes, 10 into 09. Atlas of Roots,
  empty since 2025-09-29); each memory file back-pointed, vault note canonical for the idea.
- N-now corrected in MEMORY.md: 49/84, down 11 — the counter is a rolling window.

Steward-authorized. Trials: thread-query + Smart Connections, both graded 2026-10-05.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQKeKY9T9d95KpvHwwok8T
2026-08-24 11:25:39 +02:00

3.8 KiB

thread-query.py — pre-registered trial

Built and wired 2026-08-24 on steward authorization. Pre-registered before it runs, so the verdict is not written afterwards to match whatever happened.

What it is

~/dotfiles/scripts/thread-query.py, invoked from /wake-up §2.b.3 with --log. It queries the memory corpus and the vault using the pulling thread, excluding the recency slice the wake already loads and favouring older material. Returns pointers and literal matching lines, never summaries.

Why

Measured 2026-08-24: /wake-up reads 6,107 words of an 859,803-word memory corpus — 0.71%, always the newest. Everything else is dark unless someone goes looking. Twice that day someone did, by hand, and both greps produced the session's best findings (the Hearth definition in a December Compass document; the humic layer from April). This turns that hand-motion into an instrument.

The step was previously described in the skill ("grep the memory dir…") and fired almost never — the same described-not-invoked gap that had the link canary hand-typed twice.

Motion comes from the current work, not the calendar. Matuschak's report is that scheduled resurfacing fails because you are thinking about something else when it arrives.

⚠ Known weakness, stated before the trial rather than after

Live run 2026-08-24: results are plausible but noisy. Generic terms in a thread — vault, instruments, conventions — are this corpus's own background vocabulary and pull weak matches. Threads phrased with distinctive terms do markedly better. No further blind tuning: more adjustment without evidence is guessing, and the trial is the evidence.

What was already caught, and how

PASS-BUT-FALSELY on first live run, at 14/14 selftests green. Scoring on distinct terms present anywhere in a document made Carruthers' Book of Memory (1.6 MB) and Yates' Art of Memory (1 MB) the top hits for every query — a million-word book contains "instruments" and "heap" by accident. Presence over an unbounded document measures length, not relevance. The fixture held only small uniform files, so the selftest certified its author's blind spot — the identical shape vault-links.py hit on 2026-08-23.

Fixed by scoring on distinct terms co-occurring inside a ~40-line window. A length-bias control is now in the selftest (16 checks).

Also caught: the first archive fix reintroduced the 2026-08-23 substring bug, and was replaced by adopting vault-links.py's rule rather than writing a second one.

Pre-registered questions — grade 2026-10-05

File the result whichever way it falls. Grade from thread-query-log.jsonl, not from memory — a trial graded by recollection has the same shape as the obligation that failed the daily note.

  1. How many wakes surfaced something the steward would have wanted and would not otherwise have found? Count them. Zero is a real answer.
  2. Was any of it dormant — older than three months and not already linked from the thread? Recent-and-obvious does not count; the deficit is re-encounter, not search.
  3. What fraction of runs returned nothing? A tool that returns something every time is probably returning noise.
  4. Did the two-pointer cap ever cut something better than what was shown?

Falsifier. If (1) is zero or near-zero, the step comes out of /wake-up. If (3) is near zero while (1) is low, the scoring is returning noise and the tool comes out. A trial that can only conclude "keep it" is not a trial.

  • 97. Tools and Systems/Smart Connections — six-week trial.md — the while-you-write half, same grading date. This is the at-the-wake half.
  • 08. Notes/The compost heap — putrefaction as a phase.md — the figure. This is the turning.