David/dotfiles
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[FIX] The link canary was blind to code spans, to wikilinks, and to its own class

Browse Source 
The steward's 2026-08-09 to-do read "the gap is neither knowledge nor home but
the absence of an EXECUTABLE." The premise was false: classify_pointers has
existed since 19bddd5 (2026-08-08), wired to SessionStart, with controls. The
gap was that the executable was incomplete, and the incompleteness had already
produced a false positive.

Four defects, three named in the spec and one found by building it:

1. CODE SPANS. `](file.md)` inside backticks read as a pointer, so the single
   DEAD pointer reported on 2026-08-09 was the link pattern written inside
   MEMORY.md's own specification of this canary. An instrument that flags its
   own documentation flags it every wake forever, and the real signal drowns —
   the same "known canary bug" dismissal the 2026-07-28 block was written to
   end, arriving by a second route. Fences and inline spans are blanked with
   offsets preserved; inline spans may not cross a newline and an unterminated
   fence does not match, so a stray backtick can never blank the file and HIDE
   dead pointers.

2. WIKILINKS. reference-verification-ladder.md has specified this canary as
   covering "every `](file.md)` and `[[wikilink]]`" since 2026-07-06. Only the
   first half was ever built. 31 wikilinks now checked.

3. BREAKAGE AGE, derived from git rather than a stored prior run — a state file
   would make this the one cached section in a digest whose governing property
   is that it is computed. Where git cannot answer, it says so.

4. Found by running it: the first wikilink pass reported only UNWRITTEN, and
   both live hits were [[trust-prior-pass-frame]], whose file EXISTS as
   feedback-trust-prior-pass-frame.md. That is precisely the one-word alarm the
   comment ten lines above it was written to forbid. Wikilinks now report three
   outcomes and hand back the replacement slug. Both are repaired here.

The wake-up skill and the ladder now POINT AT the executable instead of
describing the check — the described-not-invoked gap is why it kept being
retyped by hand on 2026-08-08 and 2026-08-09.

Verify: python3 scripts/wake-digest.py --selftest   (61 checks, exit 0)
        python3 scripts/wake-digest.py | grep 'MEMORY POINTERS'
Induced red: blank_code reverted to a no-op (behaviour, not the symbol) →
        exit 2, five named failures, no traceback; direction controls held.

Not changed: the wrap_inside detector, which announced "PREVIOUS SESSION DID
NOT WRAP" for a session that wrapped at 19:48 and kept working until 21:54 —
a two-valued detector over a three-case state. Named in the ledger, not fixed.
This commit is contained in:
David F Glidden
2026-08-10 16:01:51 +02:00
parent ed9951d1d7
commit 32b0c9dfc6
5 changed files with 255 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/wake-digest.py
+201 -5
View File
@@ -429,10 +429,52 @@ def brief_age_days():
# memory files' own prose (17 uses). Absolute still VALIDATES here — it is not
# wrong, only unportable — but it is counted and reported so it cannot re-enter
# silently.
#
# 2026-08-10 — three completions, all earned before this code existed.
#
# (1) CODE SPANS. The matcher read `](file.md)` inside backticks as a pointer,
# so on 2026-08-09 it reported exactly one DEAD pointer: the link pattern
# written inside MEMORY.md's own SPECIFICATION OF THIS CANARY. An
# instrument that flags its own documentation flags it every wake, forever,
# and the real signal drowns in a permanent known-false line — the same
# "known canary bug" dismissal the block above was written to end, arriving
# by a second route. Code spans and fenced blocks are blanked before
# matching, offsets preserved so every reported line number stays true.
# Inline spans may not cross a newline: a stray unmatched backtick must
# bound its damage to one line rather than blanking the rest of the file
# and HIDING dead pointers. An unterminated fence simply does not match,
# so the failure direction is over-reporting, never silence.
#
# (2) WIKILINKS. `reference-verification-ladder.md` has specified the canary as
# covering "every `](file.md)` and `[[wikilink]]`" since 2026-07-06; only
# the first half was ever built. Reported as UNWRITTEN, not DEAD, because
# ~/CLAUDE.md rules a wikilink with no file yet to be legitimate — it marks
# something worth writing. A permitted forward-reference and a broken index
# pointer are different findings and must not share a word.
#
# (3) PRE-EXISTING vs NEWLY BROKEN. The 2026-08-09 spec asked for it. Derived
# from git, NOT from a stored prior run: a state file would make this the
# one cached section in a digest whose governing property is that it is
# computed. Where git cannot answer — a target outside this repo — it says
# so rather than guessing.
NONPORTABLE_RE = re.compile(r"^/(Users|home)/")
MEM_INDEXES = ["MEMORY.md", "MEMORY-reference.md"]
POINTER_RE = re.compile(r"\]\(([^)\s]+\.md)\)")
WIKILINK_RE = re.compile(r"\[\[([^\[\]|#\n]+?)(?:#[^\[\]|\n]*)?(?:\|[^\[\]\n]*)?\]\]")
FENCE_RE = re.compile(r"^(?P<f>`{3,}|~{3,})[^\n]*\n.*?^(?P=f)[^\n]*$", re.M | re.S)
INLINE_CODE_RE = re.compile(r"`+[^`\n]*`+")
def blank_code(text):
"""Blank code-span and fenced-block CONTENT, preserving offsets and newlines.
Preserving offsets is the point: every line number this module reports is
computed from the blanked text, so it must still address the real line.
"""
def blank(m):
return "".join("\n" if c == "\n" else " " for c in m.group(0))
return INLINE_CODE_RE.sub(blank, FENCE_RE.sub(blank, text))
def find_basename(name):
@@ -458,6 +500,7 @@ def resolve(raw, base):
def classify_pointers(text, base):
"""(n_checked, mis_authored, dead, nonportable) — outcomes, never one word."""
n, mis, dead, nonport = 0, [], [], []
text = blank_code(text)
for m in POINTER_RE.finditer(text):
raw = m.group(1)
if raw.startswith(("http://", "https://", "mailto:")):
@@ -479,8 +522,93 @@ def classify_pointers(text, base):
return n, mis, dead, nonport
def near_slugs(slug, memdir):
"""Memory files this wikilink was plainly REACHING FOR. Prefix-family only.
`[[trust-prior-pass-frame]]` wants `feedback-trust-prior-pass-frame.md`; the
slug is the tail of the real stem after a hyphen. Matching on that boundary
(never on a bare substring) keeps `[[arc]]` from claiming every file with
"arc" inside a word.
"""
out = []
try:
stems = [f[:-3] for f in os.listdir(memdir) if f.endswith(".md")]
except OSError:
return out
for stem in stems:
if stem.endswith("-" + slug) or slug.endswith("-" + stem):
out.append(stem)
return sorted(out)
def classify_wikilinks(text, memdir):
"""(n_checked, mis_authored, unwritten) — three outcomes, never one word.
The same discipline as the path pointers above, and for the same reason: the
first build of this function reported only UNWRITTEN, and its two live hits
were both `[[trust-prior-pass-frame]]`, whose file EXISTS as
`feedback-trust-prior-pass-frame.md`. One word would have sent a fixable
typo to the wake every morning wearing the label "nothing to do here".
UNWRITTEN, never DEAD, for the genuine misses: ~/CLAUDE.md rules that a
`[[name]]` matching no file yet "is fine — it marks something worth writing
later, not an error." A permitted forward-reference is not a defect.
"""
n, mis, unwritten = 0, [], []
text = blank_code(text)
for m in WIKILINK_RE.finditer(text):
slug = m.group(1).strip()
if not slug:
continue
n += 1
stem = slug[:-3] if slug.endswith(".md") else slug
if os.path.exists(os.path.join(memdir, stem + ".md")):
continue
line = text.count("\n", 0, m.start()) + 1
near = near_slugs(stem, memdir)
if near:
mis.append((line, slug, " | ".join(near)))
else:
unwritten.append((line, slug))
return n, mis, unwritten
# Breakage age is DERIVED from git, never from a stored prior run — a cache is
# the one thing this digest's governing property forbids. Two questions, both
# answerable at HEAD: was the pointer already written, and did its target
# already exist? Their four combinations are the four honest verdicts.
def breakage(raw, head_text, target_at_head):
"""PRE-EXISTING vs NEWLY BROKEN. `None` inputs mean 'git cannot say'."""
if head_text is None:
return "age unknown — no committed version of this index to compare"
if raw not in head_text:
return "NEW — this pointer was added since HEAD"
if target_at_head is True:
return "NEWLY BROKEN — target existed at HEAD and is gone now"
if target_at_head is False:
return "pre-existing — already broken at HEAD"
return "pre-existing at HEAD — target lives outside this repo, break date unknown"
def head_text_of(relpath):
"""The index as committed. None => never committed, or git unavailable."""
return sh(["git", "-C", D, "show", f"HEAD:{relpath}"])
def target_at_head(raw, memrel):
"""Did the pointer's target exist at HEAD? None where git cannot know."""
base = os.path.basename(raw)
if os.path.normpath(resolve(raw, os.path.realpath(MEM))) != os.path.normpath(
os.path.join(os.path.realpath(MEM), base)):
return None # leaves the memory dir — outside git's reach here
return sh(["git", "-C", D, "cat-file", "-e", f"HEAD:{memrel}/{base}"]) is not None
def sec_pointers():
total, mis, dead, nonport = 0, [], [], []
wtotal, wmis, unwritten = 0, [], []
memdir = os.path.realpath(MEM)
memrel = os.path.relpath(memdir, D)
for name in MEM_INDEXES:
path = os.path.realpath(os.path.join(MEM, name))
text = read(path)
@@ -489,10 +617,16 @@ def sec_pointers():
continue
n, m_, d_, np_ = classify_pointers(text, os.path.dirname(path))
total += n
head = head_text_of(f"{memrel}/{name}")
mis += [(name,) + t for t in m_]
dead += [(name,) + t for t in d_]
dead += [(name, ln, raw, breakage(raw, head, target_at_head(raw, memrel)))
for ln, raw, _ in d_]
nonport += [(name,) + t for t in np_]
return total, mis, dead, nonport
wn, wm, wu = classify_wikilinks(text, memdir)
wtotal += wn
wmis += [(name,) + t for t in wm]
unwritten += [(name,) + t for t in wu]
return total, mis, dead, nonport, wtotal, wmis, unwritten
# ---------- stray maps ----------
@@ -609,6 +743,61 @@ def selftest():
classify_pointers("[x](https://a.md)", MEMDIR) == (0, [], [], []))
chk("classify_pointers returns clean on empty input",
classify_pointers("", MEMDIR) == (0, [], [], []))
NOWHERE = "no-such-file-anywhere-xyzzy-9931.md"
print("\ncode spans [2026-08-09: the canary flagged its own specification]:")
chk("a pointer INSIDE a code span is not a pointer",
classify_pointers(f"`[y]({NOWHERE})`", MEMDIR) == (0, [], [], []))
chk("...and the SAME pointer outside one still fires [direction control —"
" blanking must not swallow everything]",
classify_pointers(f"[y]({NOWHERE})", MEMDIR)[0] == 1)
chk("MEMORY.md's own canary spec is silent [the exact 2026-08-09 false positive]",
(lambda t: classify_pointers(t, MEMDIR) == (0, [], [], [])
and classify_wikilinks(t, MEMDIR) == (0, [], []))(
"→ `~/dotfiles/scripts/`, invoked not described; `](file.md)` +"
" `[[wikilink]]` over both memory indexes"))
chk("blanking preserves LINE NUMBERS [offsets kept, not deleted]",
classify_pointers(f"`](x.md)`\n[y]({NOWHERE})\n", MEMDIR)[2][0][0] == 2)
chk("a fenced block is blanked",
classify_pointers(f"```\n[y]({NOWHERE})\n```\n", MEMDIR) == (0, [], [], []))
chk("a STRAY unmatched backtick blanks nothing [damage bounded to one line;"
" a greedy matcher would HIDE dead pointers]",
classify_pointers(f"a ` b\n[y]({NOWHERE})\n", MEMDIR)[0] == 1)
chk("an UNTERMINATED fence blanks nothing [fails toward reporting, not silence]",
classify_pointers(f"```\n[y]({NOWHERE})\n", MEMDIR)[0] == 1)
print("\nwikilinks [the ladder specified them 2026-07-06; never built until now]:")
chk("a wikilink whose file exists resolves",
classify_wikilinks("see [[MEMORY]]", MEMDIR) == (1, [], []))
chk("a wikilink with no file is UNWRITTEN, not dead [~/CLAUDE.md rules it legitimate]",
(lambda r: r[0] == 1 and not r[1] and len(r[2]) == 1)(
classify_wikilinks("see [[no-such-memory-xyzzy-9931]]", MEMDIR)))
chk("a NEAR-MISS slug is MIS-AUTHORED with the real file handed back"
" [live: the two hits the one-word version mislabelled]",
(lambda r: len(r[1]) == 1 and not r[2]
and r[1][0][2] == "feedback-trust-prior-pass-frame")(
classify_wikilinks("[[trust-prior-pass-frame]]", MEMDIR)))
chk("...and a genuine miss does NOT collapse into mis-authored [direction control]",
not classify_wikilinks("[[no-such-memory-xyzzy-9931]]", MEMDIR)[1])
chk("near_slugs matches on the HYPHEN boundary, not bare substring"
" [else [[arc]] would claim every file with 'arc' in a word]",
all(s.endswith("-arc") or "arc".endswith("-" + s) for s in near_slugs("arc", MEMDIR)))
chk("an alias and a heading are stripped down to the slug",
classify_wikilinks("[[MEMORY#Index|the index]]", MEMDIR) == (1, [], []))
chk("a wikilink inside a code span is not a wikilink",
classify_wikilinks("`[[no-such-memory-xyzzy-9931]]`", MEMDIR) == (0, [], []))
chk("classify_wikilinks returns clean on empty input [positive control]",
classify_wikilinks("", MEMDIR) == (0, [], []))
print("\nbreakage age [derived from git — this digest may not cache]:")
chk("a pointer absent at HEAD is NEW",
breakage("x.md", "nothing here", False).startswith("NEW"))
chk("a pointer present at HEAD whose target was there too is NEWLY BROKEN",
breakage("x.md", "[a](x.md)", True).startswith("NEWLY BROKEN"))
chk("a pointer present at HEAD whose target was already gone is pre-existing",
breakage("x.md", "[a](x.md)", False).startswith("pre-existing"))
chk("an out-of-repo target says the break date is unknown [honest degradation]",
"unknown" in breakage("x.md", "[a](x.md)", None))
chk("no committed index reads as UNKNOWN, never as pre-existing"
" [negative control — absence of evidence]",
breakage("x.md", None, None).startswith("age unknown"))
chk("is_homed TRUE for a Desktop map symlinked into maps/ [live substrate]",
is_homed(os.path.join(DESKTOP, "corpus-index-2026-07-26.md"), MAPS_DIR))
chk("is_homed FALSE for a regular file [negative control — must not pass everything]",
@@ -676,16 +865,23 @@ def main():
o.append(f"\nGOVERNANCE DRIFT — ~/CLAUDE.md: {drift} substrate-contradicted claim(s)"
" (detection only; correction needs [ESCALATE])")
ptot, pmis, pdead, pnp = sec_pointers()
ptot, pmis, pdead, pnp, wtot, wmis, wun = sec_pointers()
o.append(f"\nMEMORY POINTERS — {ptot} checked · {len(pmis)} mis-authored · {len(pdead)} dead"
+ (f" · {len(pnp)} non-portable" if pnp else ""))
+ (f" · {len(pnp)} non-portable" if pnp else "")
+ f" | {wtot} wikilinks · {len(wmis)} mis-authored · {len(wun)} unwritten")
for f, ln, raw, fix in pmis:
o.append(f" MIS-AUTHORED {f}:{ln} {raw}")
o.append(f" → target exists; replace with: {fix}")
for f, ln, raw, _ in pdead:
for f, ln, raw, age in pdead:
o.append(f" DEAD {f}:{ln} {raw} (no file of that name in ~/_Dev or the memory dir)")
o.append(f" → {age}")
for f, ln, raw in pnp:
o.append(f" NON-PORTABLE {f}:{ln} {raw} (resolves, but hardcodes this machine)")
for f, ln, slug, near in wmis:
o.append(f" MIS-AUTHORED {f}:{ln} [[{slug}]]")
o.append(f" → the file exists; replace with: [[{near}]]")
for f, ln, slug in wun:
o.append(f" UNWRITTEN {f}:{ln} [[{slug}]] (permitted forward-reference, not an error)")
strays = stray_maps()
if strays: