[HARDENING] governance-mcp: keyword search — the jurist can discover an item whose id it does not know (PENDING-86 d)

Steward-authorized 2026-08-05, completing the (a)+(d) pair the jurist asked for.

The failure this closes is NOT "cannot read item X" — (a) fixed that. It is
"cannot DISCOVER item X whose id it does not already know": the 2026-07-29 case
where a ruling demanded an outcome REVIEWED-74 had settled four days earlier, in
a file the jurist could read but had no reason to open. Keyed retrieval cannot
serve that; only search can.

`governance_search(query, limit)` over PENDING / PENDING-archive / REVIEWED.
Result unit is the ITEM, boundaries from wd.item_spans — no second definition of
"an item" (the 2026-07-28 bug that hid twenty). Results name ids to hand to
governance_item, so the two tools compose.

Three deliberate properties:

  - Terms are ANDed, and that is DISCLOSED on every result. A silently
    conjunctive matcher is exactly how recall dies as a question lengthens —
    found in the engine yesterday (PENDING-97, "what does levi mean by the gray
    zone" -> 0 over ten real matches). The same shape is not being rebuilt here
    unannounced.
  - A miss is a legible empty: it states the corpus, the item count scanned, the
    terms, and the match mode, and says outright that a longer query narrows
    fast. Silence discloses its own blindness (PENDING-96's discipline, applied
    to a new instrument on the day it was ruled).
  - Ranked by exact-phrase then raw term-count, labelled as a term COUNT and not
    a relevance score — it is a field this code actually computes.

Plus a query-INDEPENDENT structural pass: an item header hidden by leading
whitespace is invisible to item_spans, so it can never appear in results and its
absence reads as a genuine miss. Such headers are now reported beside the
results. An earlier draft flagged any uncovered matching line and drowned the
signal in each file's preamble — which is how a warning stops being read.

That pass earned itself immediately: REVIEWED-11, REVIEWED-12 and REVIEWED-74
were all indented and therefore unreachable by governance_item. REVIEWED-74 is
precisely the ruling the jurist could not find, so its failure was
over-determined — it did not know the id, AND the id would not have worked.
Steward unindented all three (REVIEWED.md is his file, not the executor's, per
Constitutional Constraint 1); items visible 78 -> 81, hidden headers now zero.

Selftest 35 -> 44 controls, 0 fail, including a negative control that goes red
if a header is ever hidden again. Live stdio round-trip confirms six tools and a
correct search result.

⚠ Requires a Claude.app restart to expose the new tool.

Refs PENDING-86 (d), PENDING-82, PENDING-96, PENDING-97.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AB3Kryoy6b1pm2Nz1DYdLh

scripts/governance-mcp.py

@@ -36,8 +36,10 @@ Provenance: 2026-07-28, on PENDING-81's MCP leg. Sibling of wake-digest.py and
 governance-drift-check.py; imports the first.
 """
 import importlib.util
+import inspect
 import json
 import os
+import re
 import subprocess
 import sys
 import time
@@ -160,6 +162,107 @@ def t_read(args):
             + "\n".join(chunk) + more)
 
 
+SEARCH_KEYS = ("pending", "pending-archive", "reviewed")
+
+
+def t_search(args):
+    """Keyword search across the authorization corpus — PENDING-86 option (d).
+
+    The failure this exists for is NOT 'the jurist cannot read item X'. It is
+    'the jurist cannot DISCOVER item X whose id it does not already know' — the
+    2026-07-29 case where a ruling demanded an outcome REVIEWED-74 had settled
+    four days earlier, in a file the jurist could read but had no reason to open.
+    Keyed retrieval cannot fix that; only search can.
+
+    Three deliberate properties:
+      * The result unit is the ITEM, because the jurist's next move is
+        governance_item(id). Boundaries come from wd.item_spans — no second
+        definition of 'an item' (the 2026-07-28 bug).
+      * Terms are ANDed, and that is disclosed, because a silently conjunctive
+        matcher is exactly how recall dies as a question lengthens (the engine's
+        PENDING-97, found 2026-08-04). A zero-result search says what it
+        searched and how, so silence is legible rather than bare.
+      * A raw-text pass runs BESIDE the item pass and reports matches lying
+        outside every span item_spans can see — an indented header, a region
+        the parser drops. Search reveals the hole instead of papering it.
+    """
+    q = (args.get("query") or "").strip()
+    if not q:
+        return "ERROR: query is required, e.g. 'order attestation' or 'two-column'."
+    terms = [t for t in q.lower().split() if t]
+    lim = max(1, min(int(args.get("limit") or 12), 50))
+
+    hits, scanned, orphan_lines = [], 0, []
+    for key in SEARCH_KEYS:
+        text = read(FILES[key][0])
+        if text is None:
+            continue
+        fname = os.path.basename(FILES[key][0])
+        lines = text.split("\n")
+        covered = set()
+        for head, start, end in wd.item_spans(text):
+            scanned += 1
+            covered.update(range(start, end))
+            body = "\n".join(lines[start - 1:end - 1])
+            low = body.lower()
+            if not all(t in low for t in terms):
+                continue
+            n = sum(low.count(t) for t in terms)
+            phrase = q.lower() in low
+            snippet = ""
+            for ln in body.split("\n"):
+                if any(t in ln.lower() for t in terms):
+                    snippet = " ".join(ln.split())[:200]
+                    break
+            hits.append((phrase, n, head, fname, start, snippet))
+        # Structural health pass, deliberately INDEPENDENT of the query: an item header
+        # hidden by leading whitespace is invisible to item_spans, so it can never appear
+        # in results and its absence would look like a genuine miss. Query-independent
+        # because the jurist needs to know the index is incomplete whatever it searched.
+        # Narrow by construction — an earlier draft flagged any uncovered matching line
+        # and drowned the signal in each file's preamble, which is exactly how a warning
+        # stops being read.
+        infence = False
+        for i, ln in enumerate(lines, 1):
+            if ln.lstrip().startswith("```"):
+                infence = not infence
+                continue
+            if infence or i in covered:
+                continue
+            m = re.match(r"^\s+## ((?:PENDING|REVIEWED|COMPLETED)-\S+)", ln)
+            if m:
+                orphan_lines.append((fname, i, " ".join(ln.split())[:160]))
+
+    hits.sort(key=lambda h: (not h[0], -h[1]))
+    mode = ("exact phrase, else all-terms-present" if len(terms) > 1
+            else "single term, substring")
+    o = [f"[governance_search {q!r} — {len(hits)} matching item(s) of {scanned} scanned "
+         f"across {', '.join(SEARCH_KEYS)}]",
+         f"match mode: ALL {len(terms)} term(s) must appear in the same item "
+         f"({mode}); ranked by exact-phrase first, then raw term-count — "
+         f"term-count is a computed field, not a relevance score.", ""]
+    if not hits:
+        o.append("NO ITEM MATCHED. This is a legible empty, not a claim that the corpus "
+                 "is silent on the subject: the matcher is CONJUNCTIVE, so a longer query "
+                 "narrows fast. Retry with fewer or more common terms before concluding "
+                 "nothing is on file.")
+    for phrase, n, head, fname, start, snip in hits[:lim]:
+        o.append(f"{'★ ' if phrase else '  '}{head}   [{fname}:{start}]  {n} hit(s)")
+        if snip:
+            o.append(f"      … {snip}")
+    if len(hits) > lim:
+        o.append(f"\n[… {len(hits) - lim} more — raise limit (max 50)]")
+    if orphan_lines:
+        o.append("\n⚠ MATCHES OUTSIDE ANY ITEM THE PARSER CAN SEE — a header hidden by "
+                 "leading whitespace, or a region item_spans drops. governance_item() "
+                 "cannot retrieve these by id; they are reported here so the gap is "
+                 "visible rather than silent:")
+        for fname, i, ln in orphan_lines[:10]:
+            o.append(f"      {fname}:{i}  {ln}")
+    o.append("\nNext: governance_item(id) for the verbatim body of any hit above.")
+    return "\n".join(o)
+
+
 def t_drift(_args):
     """Full drift-check output. Reports its own non-verification rather than a clean bill."""
     out = wd.sh([sys.executable, wd.DRIFT], timeout=30)
@@ -207,6 +310,17 @@ TOOLS = [
          "offset": {"type": "integer", "description": "0-based first line (default 0)."},
          "limit": {"type": "integer", "description": "Lines to return, max 2000 (default 400)."}},
       "required": ["file"]}),
+    ("governance_search", t_search,
+     "Keyword search across PENDING.md, PENDING-archive.md and REVIEWED.md. Use this to "
+     "DISCOVER a relevant prior item or ruling whose id you do not already know — the case "
+     "governance_item cannot serve, because it needs the id up front. Terms are ANDed "
+     "within a single item; results name ids to pass to governance_item.",
+     {"type": "object", "properties": {
+         "query": {"type": "string",
+                   "description": "Words that must all appear in the same item, e.g. "
+                                  "'order attestation' or 'two-column'. Not a regex."},
+         "limit": {"type": "integer", "description": "Max items to list, max 50 (default 12)."}},
+      "required": ["query"]}),
     ("drift_report", t_drift,
      "Full governance-drift-check output: claims in CLAUDE.md the substrate contradicts. "
      "Detection only — correcting doctrine requires steward authorization.",
@@ -386,6 +500,31 @@ def selftest():
     chk("a first-page read lands in the SUPERSEDED region  [negative control: proves the "
         "trap is real, not hypothetical]",
         "(obsoleted)" in t_read({"file": "chamber-spec", "limit": 300}))
+    print("\nPENDING-86 (d) — discovery without knowing the id:")
+    _s = t_search({"query": "added-side fabrication"})
+    chk("search finds REVIEWED-74 from CONTENT alone  [the 2026-07-29 failure: the jurist "
+        "demanded an outcome this ruling had settled four days earlier, and could not find it]",
+        "REVIEWED-74" in _s)
+    chk("search spans the archive too, not just open items",
+        "PENDING-archive.md" in _s)
+    chk("results carry ids to hand to governance_item  [the two tools compose]",
+        "governance_item(id)" in _s)
+    _e = t_search({"query": "zzzq nonexistent phrase"})
+    chk("a miss is a LEGIBLE empty, not a bare one  [it discloses the conjunctive matcher, "
+        "the failure shape that killed engine recall — PENDING-97]",
+        "0 matching item(s)" in _e and "CONJUNCTIVE" in _e)
+    chk("the match mode is stated on EVERY result, hit or miss  [positive control for the above]",
+        "match mode:" in _s and "match mode:" in _e)
+    chk("empty query refused rather than matching everything",
+        t_search({"query": "   "}).startswith("ERROR: query is required"))
+    chk("search takes no path and no file key — corpus is a fixed tuple",
+        set(SEARCH_KEYS).issubset(FILES) and len(SEARCH_KEYS) == 3)
+    chk("no second definition of 'an item' — search reuses wd.item_spans  [the 07-28 bug]",
+        "wd.item_spans" in inspect.getsource(t_search))
+    chk("orphan pass reports NOTHING today  [the three indented headers were unhidden "
+        "2026-08-05; this control goes red if a header is ever hidden again]",
+        "MATCHES OUTSIDE ANY ITEM" not in t_search({"query": "the"}))
+
     chk("repo_activity refuses an unlisted repo",
         t_repo({"repo": "/etc"}).startswith("ERROR: unknown repo"))
     chk("repo_activity ACCEPTS a listed repo  [positive control]",