David F GliddenandClaude Opus 5
3a1790dcd6
Closes the scope gap census 01 declared for itself: the seven instruments it named as uncensused. Pre-registered before any source or config was read, with predictions and a discrimination condition. Census 01 asked whether an instrument had a real negative instance — a question about CAPABILITY. Census 02 asks whether it has ever engaged in real life. Those come apart exactly at the drift-checker's shape, and 2026-08-04 found the gap six times (retrieval_count = 0 across 19,915 nodes for four months; two replay modules that have never processed an event). VERDICT: every instrument a human runs by hand has a rich firing record; every instrument that runs by itself has none — and the two guarding the engine's output have no consumer at all. The record divides by whether a human is in the invocation path, not by age, quality, or importance. verify-before-compose fired exactly twice (2026-07-17, 2026-07-18), evidence surviving only in harness transcripts; and it CANNOT fire on 31 of 59 guarded files, including the live constitution, because it folds the existing file's contents into its search for the attestation. audit_cruft, verify_conversion and apply_char_glyphs are exemplary. resolve_archived_source is healthy at 349/349 and has zero log entries. studium verify-quote and fidelity_equivalence@2 have no production call site at all. Prediction 5 inverted for the second census running, for a new reason. Census 01: decay, not construction, is the failure mode. Census 02: the recording is attached to the human, so an instrument's record vanishes the moment it is automated — which is when it starts running often enough to matter. Two of my own candidate findings died to their controls and are recorded as such: probing the resolver with engine source_ids against the chamber's canonical_slug key space (one sentence from "the resolver is inert"), and reading character_as_image at the wrong YAML nesting (nearly "zero glyph maps declared"; there are two sources and a 63-item census). Filed together: PENDING-95 [HARDENING] the hook cannot fire on the constitution · PENDING-96 [HARDENING] "SILENCE — ✓ warranted" certifies the index and claims the answer · PENDING-97 [PROPOSAL] FTS AND-s bare tokens with no semantic layer, recall dies as questions lengthen · PENDING-98 [HARDENING] firing history exists only where a human invokes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
78 lines
8.8 KiB
Markdown
78 lines
8.8 KiB
Markdown
# Census 02 — RESULT
|
||
|
||
**Run:** 2026-08-04. Pre-registered at `census-02-have-they-ever-fired-PREREGISTRATION.md` before any instrument's source or config was read.
|
||
**Question:** has each instrument ever produced a positive result — ever actually fired — and does a record exist that could tell us either way?
|
||
|
||
---
|
||
|
||
## Verdict in one line
|
||
|
||
**Every instrument that a human runs by hand has a rich firing record. Every instrument that runs by itself has none — and the two that guard the engine's output have no consumer at all.**
|
||
|
||
The census discriminated (both buckets populated, per the pre-registered condition), so it is reportable as a result rather than as nothing.
|
||
|
||
---
|
||
|
||
## The seven, censused entire
|
||
|
||
| # | Instrument | Axis A — engagement | Axis B — firing record | Verdict |
|
||
|---|---|---|---|---|
|
||
| 1 | `verify-before-compose` | **LIVE**; blocks a new ungrounded artifact (exit 2, produced today). **Cannot fire on 31 of 59 guarded files**, including the live constitution | 2 firings — 2026-07-17 09:44:34 (Write), 2026-07-18 11:55:44 (Edit). Recoverable **only** from Claude Code session transcripts | **FIRED-UNRECORDED** |
|
||
| 2 | `audit_cruft` | LIVE, wired, fleet-tested | Dense: 2026-06-16 `PASS-BUT-FALSELY` ×2 → **160 corpus files carrying residue the old gate was blind to** (95 image embeds, 64 raw svg, 8 `U+FFFD`); 2026-07-19 kindle-residue class | **FIRED-RECORDED** |
|
||
| 3 | `verify_conversion` | LIVE, wired, fleet-tested | 2026-06-16 calibration — **948/952 pass on the known-clean Loeb tier, 4 genuine truncated stubs**; false-failed a clean Loeb Aeschylus → fixed; 2026-06-28 `PASS-BUT-FALSELY` in the gate itself | **FIRED-RECORDED** |
|
||
| 4 | `resolve_archived_source` | **LIVE and healthy** — 349/349 manifest entries keyed; positive-controlled: real slugs resolve and the files exist on disk | **Zero mentions in `tool-evolution-log.md`.** It runs on every graduation and has never once been reviewed after use | **FIRED, UNLOGGED** |
|
||
| 5 | `apply_char_glyphs` | LIVE, fleet-tested, REFUSE teeth intact; 2 sources mapped, 63-item census | 2026-07-23 — Levi, **527 docs, 0 unclassified, 0 residual glyph `<img>`**, recorded in both the log and the spec's declared data | **FIRED-RECORDED** |
|
||
| 6 | studium `verify-quote` V1 | Functional — **42/42** deliberate-mismatch checks pass | **No production call site anywhere.** Only its own CLI `main()` and its test suite | **NO RECORD CAN TELL** |
|
||
| 7 | `fidelity_equivalence@2` | Shipped and correct (`FIDELITY_VERSION = "fidelity_equivalence@2"`) | Imported by `verify_quote` alone — it fires exactly as often as #6 does, which is unrecorded | **NO RECORD CAN TELL** |
|
||
|
||
---
|
||
|
||
## The finding
|
||
|
||
**The tool-evolution log is the only mechanism in this system that successfully records firing history — and it only reaches tools a human runs by hand and then reviews.**
|
||
|
||
Where the discipline reaches (#2, #3, #5), the record is exemplary: dated, artifact-named, `PASS-BUT-FALSELY` recorded as the priority signal, tool patched and the reason logged. This is the strongest instrument record anywhere in the system, stronger than the verification ladder's and stronger than `fool/`'s.
|
||
|
||
Where it does not reach, nothing records:
|
||
|
||
- `verify-before-compose` fires inside the harness, so no human is present to log it. Its two firings survive only because Claude Code happens to retain transcripts — an accident of tooling with unknown retention, not a designed record.
|
||
- `resolve_archived_source` runs inside `graduate_to_canonical` on every graduation. Because a human never *invokes* it, nobody ever *reviews* it. Zero log entries for a function that has run hundreds of times.
|
||
- `verify-quote` and `fidelity_equivalence@2` are invoked by nothing at all.
|
||
|
||
The rule the log states — *"after **every** use — success or failure — we review how the tool actually behaved"* — is in practice **after every use a human initiates.** Automatic use is invisible to it by construction.
|
||
|
||
## Two structural findings, beyond the record
|
||
|
||
**1. `verify-before-compose` is disarmed on the artifact it exists to protect.** Lines 40–41 fold the *existing file's* contents into the search for the grounding attestation. Once a file contains `GROUNDED-IN:` anywhere, no future write to it can ever be gated. **31 of 59 guarded files are in that state**, including `chamber-library-specification.md`, every versioned spec v2.1.0–v2.9.0, and `graduation-spec.yaml`. The behaviour is documented ("or the existing file"; "a speed-bump… not a guarantee"); the *consequence* — coverage decaying monotonically to zero, and already zero on the constitution — is documented nowhere.
|
||
|
||
Countervailing evidence, recorded because it cuts the other way: **all 28 guarded files lacking a marker are dated ≤ 2026-07-17; every constitutional artifact created after the hook landed carries one.** Consistent with the gate having shaped behaviour even where it cannot now block. Filename dates are a proxy; creation dates are not git-verified.
|
||
|
||
**2. The engine's `SILENCE — ✓ warranted` certifies index currency and presents it as an answer about the corpus.** Asked `grey zone`, the engine replies *"No match — this is genuine silence, not a gap."* The corpus holds **ten** matches for `gray zone`, **all ten in `levi-drowned-and-saved`**. The warrant checks that the index is complete and current; it cannot check that retrieval reached what is there. This is verbatim the shape that opened census 01 — a check certifying a property of the code while claiming a property of the result — now at the engine's consuming end and wearing a checkmark.
|
||
|
||
Mechanism, and it is wider than spelling: `retrieve.py:103` passes the normalized user string straight to `drawers_fts MATCH`, where bare terms are **AND-ed**. Recall collapses as the question lengthens — `gray` 51 hits, `gray zone` 10, `levi the gray zone` **0**, `what does levi mean by the gray zone` **0**. Natural-language questions — precisely what *"enter into discourse with my library"* means — return certified silence by default. There is no vector table; `embed_spike.py` / `rerank_spike.py` remained spikes.
|
||
|
||
Discrimination held here too: the probe *"the quality without a name"* is **correctly** silent — *The Timeless Way of Building* is not among the 13 sources. One real defect and one real correct silence, side by side.
|
||
|
||
## Residual findings
|
||
|
||
- **No `.glyph-manifest.json` has ever been written anywhere.** The two-sided manifest (REVIEWED-74) is the input to `resolve_added`'s added-side fabrication check; its wiring is exercised only by a synthetic fixture in `test_tools.py`. The check has never seen a real manifest.
|
||
- **The engine's `source_id` and the chamber's `canonical_slug` are disjoint key spaces for the same works** (`levi-drowned-and-saved` vs `the-drowned-and-the-saved-primo-levi`), with nothing joining them. A 7th instance of the shared-name/two-senses class the chamber CLAUDE.md counts.
|
||
- `engine/verify_quote.py:7` names `fidelity_equivalence@1` in prose while the shipped constant is `@2`. Stale docstring; `[FIX]` class.
|
||
|
||
## Two of my own claims died to their controls — recorded, because that is the census working
|
||
|
||
- I probed `resolve_archived_source` with engine `source_id`s and got `None` three times, including for the nonsense control. Reporting "the resolver is inert" was one sentence away. The positive control — resolving real slugs *drawn from the manifest itself* — showed it healthy at 349/349. **The instrument was mine, and it was wrong.**
|
||
- I read `character_as_image` at the YAML top level, got nothing, and nearly reported "zero glyph maps declared." It is nested at `promotion.character_as_image` and holds **2 sources and a 63-item census**.
|
||
|
||
Both are the day's own failure class (a number from an uncontrolled instrument), and both died to a control run before the claim left the workspace.
|
||
|
||
## Predictions, graded
|
||
|
||
1. `verify-before-compose` LIVE and fired ≥ once (~0.75). **Held** — exactly twice, both dated.
|
||
2. No durable firing record for it (~0.85). **Held.**
|
||
3. At least one of the seven INERT (~0.6). **Reframed, not held.** None is inert in the drift sense — every one still engages what it was built to engage. The real shapes were *disarmed-by-accumulation* (#1) and *no-consumer* (#6, #7), neither of which I anticipated.
|
||
4. Chamber tools FIRED-RECORDED via the log (~0.5). **Held and stronger than predicted** for #2/#3/#5 — and **wrong for #4**, which is a chamber tool with zero log entries because no human invokes it.
|
||
5. Overall skew FIRED-UNRECORDED dominant. **Wrong.** The skew is not by age or by quality; it is by **whether a human is in the invocation path**.
|
||
|
||
Prediction 5 has now inverted twice, in two censuses, for two different reasons. Census 01: decay, not construction, is the failure mode. Census 02: **the recording discipline is attached to the human, not to the instrument** — so an instrument's record vanishes the moment it is automated, which is exactly when it starts running often enough to matter.
|