[HARDENING] governance-mcp: keyword search — the jurist can discover an item whose id it does not know (PENDING-86 d)
Steward-authorized 2026-08-05, completing the (a)+(d) pair the jurist asked for.
The failure this closes is NOT "cannot read item X" — (a) fixed that. It is
"cannot DISCOVER item X whose id it does not already know": the 2026-07-29 case
where a ruling demanded an outcome REVIEWED-74 had settled four days earlier, in
a file the jurist could read but had no reason to open. Keyed retrieval cannot
serve that; only search can.
`governance_search(query, limit)` over PENDING / PENDING-archive / REVIEWED.
Result unit is the ITEM, boundaries from wd.item_spans — no second definition of
"an item" (the 2026-07-28 bug that hid twenty). Results name ids to hand to
governance_item, so the two tools compose.
Three deliberate properties:
- Terms are ANDed, and that is DISCLOSED on every result. A silently
conjunctive matcher is exactly how recall dies as a question lengthens —
found in the engine yesterday (PENDING-97, "what does levi mean by the gray
zone" -> 0 over ten real matches). The same shape is not being rebuilt here
unannounced.
- A miss is a legible empty: it states the corpus, the item count scanned, the
terms, and the match mode, and says outright that a longer query narrows
fast. Silence discloses its own blindness (PENDING-96's discipline, applied
to a new instrument on the day it was ruled).
- Ranked by exact-phrase then raw term-count, labelled as a term COUNT and not
a relevance score — it is a field this code actually computes.
Plus a query-INDEPENDENT structural pass: an item header hidden by leading
whitespace is invisible to item_spans, so it can never appear in results and its
absence reads as a genuine miss. Such headers are now reported beside the
results. An earlier draft flagged any uncovered matching line and drowned the
signal in each file's preamble — which is how a warning stops being read.
That pass earned itself immediately: REVIEWED-11, REVIEWED-12 and REVIEWED-74
were all indented and therefore unreachable by governance_item. REVIEWED-74 is
precisely the ruling the jurist could not find, so its failure was
over-determined — it did not know the id, AND the id would not have worked.
Steward unindented all three (REVIEWED.md is his file, not the executor's, per
Constitutional Constraint 1); items visible 78 -> 81, hidden headers now zero.
Selftest 35 -> 44 controls, 0 fail, including a negative control that goes red
if a header is ever hidden again. Live stdio round-trip confirms six tools and a
correct search result.
⚠ Requires a Claude.app restart to expose the new tool.
Refs PENDING-86 (d), PENDING-82, PENDING-96, PENDING-97.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AB3Kryoy6b1pm2Nz1DYdLh
This commit is contained in:
David F Glidden
co-authored by
Claude Opus 5
Claude Opus 5
parent
b97e77aebe
commit
673823961c
@@ -36,8 +36,10 @@ Provenance: 2026-07-28, on PENDING-81's MCP leg. Sibling of wake-digest.py and
|
||||
governance-drift-check.py; imports the first.
|
||||
"""
|
||||
import importlib.util
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
@@ -160,6 +162,107 @@ def t_read(args):
|
||||
+ "\n".join(chunk) + more)
|
||||
|
||||
|
||||
SEARCH_KEYS = ("pending", "pending-archive", "reviewed")
|
||||
|
||||
|
||||
def t_search(args):
|
||||
"""Keyword search across the authorization corpus — PENDING-86 option (d).
|
||||
|
||||
The failure this exists for is NOT 'the jurist cannot read item X'. It is
|
||||
'the jurist cannot DISCOVER item X whose id it does not already know' — the
|
||||
2026-07-29 case where a ruling demanded an outcome REVIEWED-74 had settled
|
||||
four days earlier, in a file the jurist could read but had no reason to open.
|
||||
Keyed retrieval cannot fix that; only search can.
|
||||
|
||||
Three deliberate properties:
|
||||
* The result unit is the ITEM, because the jurist's next move is
|
||||
governance_item(id). Boundaries come from wd.item_spans — no second
|
||||
definition of 'an item' (the 2026-07-28 bug).
|
||||
* Terms are ANDed, and that is disclosed, because a silently conjunctive
|
||||
matcher is exactly how recall dies as a question lengthens (the engine's
|
||||
PENDING-97, found 2026-08-04). A zero-result search says what it
|
||||
searched and how, so silence is legible rather than bare.
|
||||
* A raw-text pass runs BESIDE the item pass and reports matches lying
|
||||
outside every span item_spans can see — an indented header, a region
|
||||
the parser drops. Search reveals the hole instead of papering it.
|
||||
"""
|
||||
q = (args.get("query") or "").strip()
|
||||
if not q:
|
||||
return "ERROR: query is required, e.g. 'order attestation' or 'two-column'."
|
||||
terms = [t for t in q.lower().split() if t]
|
||||
lim = max(1, min(int(args.get("limit") or 12), 50))
|
||||
|
||||
hits, scanned, orphan_lines = [], 0, []
|
||||
for key in SEARCH_KEYS:
|
||||
text = read(FILES[key][0])
|
||||
if text is None:
|
||||
continue
|
||||
fname = os.path.basename(FILES[key][0])
|
||||
lines = text.split("\n")
|
||||
covered = set()
|
||||
for head, start, end in wd.item_spans(text):
|
||||
scanned += 1
|
||||
covered.update(range(start, end))
|
||||
body = "\n".join(lines[start - 1:end - 1])
|
||||
low = body.lower()
|
||||
if not all(t in low for t in terms):
|
||||
continue
|
||||
n = sum(low.count(t) for t in terms)
|
||||
phrase = q.lower() in low
|
||||
snippet = ""
|
||||
for ln in body.split("\n"):
|
||||
if any(t in ln.lower() for t in terms):
|
||||
snippet = " ".join(ln.split())[:200]
|
||||
break
|
||||
hits.append((phrase, n, head, fname, start, snippet))
|
||||
# Structural health pass, deliberately INDEPENDENT of the query: an item header
|
||||
# hidden by leading whitespace is invisible to item_spans, so it can never appear
|
||||
# in results and its absence would look like a genuine miss. Query-independent
|
||||
# because the jurist needs to know the index is incomplete whatever it searched.
|
||||
# Narrow by construction — an earlier draft flagged any uncovered matching line
|
||||
# and drowned the signal in each file's preamble, which is exactly how a warning
|
||||
# stops being read.
|
||||
infence = False
|
||||
for i, ln in enumerate(lines, 1):
|
||||
if ln.lstrip().startswith("```"):
|
||||
infence = not infence
|
||||
continue
|
||||
if infence or i in covered:
|
||||
continue
|
||||
m = re.match(r"^\s+## ((?:PENDING|REVIEWED|COMPLETED)-\S+)", ln)
|
||||
if m:
|
||||
orphan_lines.append((fname, i, " ".join(ln.split())[:160]))
|
||||
|
||||
hits.sort(key=lambda h: (not h[0], -h[1]))
|
||||
mode = ("exact phrase, else all-terms-present" if len(terms) > 1
|
||||
else "single term, substring")
|
||||
o = [f"[governance_search {q!r} — {len(hits)} matching item(s) of {scanned} scanned "
|
||||
f"across {', '.join(SEARCH_KEYS)}]",
|
||||
f"match mode: ALL {len(terms)} term(s) must appear in the same item "
|
||||
f"({mode}); ranked by exact-phrase first, then raw term-count — "
|
||||
f"term-count is a computed field, not a relevance score.", ""]
|
||||
if not hits:
|
||||
o.append("NO ITEM MATCHED. This is a legible empty, not a claim that the corpus "
|
||||
"is silent on the subject: the matcher is CONJUNCTIVE, so a longer query "
|
||||
"narrows fast. Retry with fewer or more common terms before concluding "
|
||||
"nothing is on file.")
|
||||
for phrase, n, head, fname, start, snip in hits[:lim]:
|
||||
o.append(f"{'★ ' if phrase else ' '}{head} [{fname}:{start}] {n} hit(s)")
|
||||
if snip:
|
||||
o.append(f" … {snip}")
|
||||
if len(hits) > lim:
|
||||
o.append(f"\n[… {len(hits) - lim} more — raise limit (max 50)]")
|
||||
if orphan_lines:
|
||||
o.append("\n⚠ MATCHES OUTSIDE ANY ITEM THE PARSER CAN SEE — a header hidden by "
|
||||
"leading whitespace, or a region item_spans drops. governance_item() "
|
||||
"cannot retrieve these by id; they are reported here so the gap is "
|
||||
"visible rather than silent:")
|
||||
for fname, i, ln in orphan_lines[:10]:
|
||||
o.append(f" {fname}:{i} {ln}")
|
||||
o.append("\nNext: governance_item(id) for the verbatim body of any hit above.")
|
||||
return "\n".join(o)
|
||||
|
||||
|
||||
def t_drift(_args):
|
||||
"""Full drift-check output. Reports its own non-verification rather than a clean bill."""
|
||||
out = wd.sh([sys.executable, wd.DRIFT], timeout=30)
|
||||
@@ -207,6 +310,17 @@ TOOLS = [
|
||||
"offset": {"type": "integer", "description": "0-based first line (default 0)."},
|
||||
"limit": {"type": "integer", "description": "Lines to return, max 2000 (default 400)."}},
|
||||
"required": ["file"]}),
|
||||
("governance_search", t_search,
|
||||
"Keyword search across PENDING.md, PENDING-archive.md and REVIEWED.md. Use this to "
|
||||
"DISCOVER a relevant prior item or ruling whose id you do not already know — the case "
|
||||
"governance_item cannot serve, because it needs the id up front. Terms are ANDed "
|
||||
"within a single item; results name ids to pass to governance_item.",
|
||||
{"type": "object", "properties": {
|
||||
"query": {"type": "string",
|
||||
"description": "Words that must all appear in the same item, e.g. "
|
||||
"'order attestation' or 'two-column'. Not a regex."},
|
||||
"limit": {"type": "integer", "description": "Max items to list, max 50 (default 12)."}},
|
||||
"required": ["query"]}),
|
||||
("drift_report", t_drift,
|
||||
"Full governance-drift-check output: claims in CLAUDE.md the substrate contradicts. "
|
||||
"Detection only — correcting doctrine requires steward authorization.",
|
||||
@@ -386,6 +500,31 @@ def selftest():
|
||||
chk("a first-page read lands in the SUPERSEDED region [negative control: proves the "
|
||||
"trap is real, not hypothetical]",
|
||||
"(obsoleted)" in t_read({"file": "chamber-spec", "limit": 300}))
|
||||
print("\nPENDING-86 (d) — discovery without knowing the id:")
|
||||
_s = t_search({"query": "added-side fabrication"})
|
||||
chk("search finds REVIEWED-74 from CONTENT alone [the 2026-07-29 failure: the jurist "
|
||||
"demanded an outcome this ruling had settled four days earlier, and could not find it]",
|
||||
"REVIEWED-74" in _s)
|
||||
chk("search spans the archive too, not just open items",
|
||||
"PENDING-archive.md" in _s)
|
||||
chk("results carry ids to hand to governance_item [the two tools compose]",
|
||||
"governance_item(id)" in _s)
|
||||
_e = t_search({"query": "zzzq nonexistent phrase"})
|
||||
chk("a miss is a LEGIBLE empty, not a bare one [it discloses the conjunctive matcher, "
|
||||
"the failure shape that killed engine recall — PENDING-97]",
|
||||
"0 matching item(s)" in _e and "CONJUNCTIVE" in _e)
|
||||
chk("the match mode is stated on EVERY result, hit or miss [positive control for the above]",
|
||||
"match mode:" in _s and "match mode:" in _e)
|
||||
chk("empty query refused rather than matching everything",
|
||||
t_search({"query": " "}).startswith("ERROR: query is required"))
|
||||
chk("search takes no path and no file key — corpus is a fixed tuple",
|
||||
set(SEARCH_KEYS).issubset(FILES) and len(SEARCH_KEYS) == 3)
|
||||
chk("no second definition of 'an item' — search reuses wd.item_spans [the 07-28 bug]",
|
||||
"wd.item_spans" in inspect.getsource(t_search))
|
||||
chk("orphan pass reports NOTHING today [the three indented headers were unhidden "
|
||||
"2026-08-05; this control goes red if a header is ever hidden again]",
|
||||
"MATCHES OUTSIDE ANY ITEM" not in t_search({"query": "the"}))
|
||||
|
||||
chk("repo_activity refuses an unlisted repo",
|
||||
t_repo({"repo": "/etc"}).startswith("ERROR: unknown repo"))
|
||||
chk("repo_activity ACCEPTS a listed repo [positive control]",
|
||||
|
||||
Reference in New Issue
Block a user