docs(governance): census 02 — has each instrument ever fired? + PENDING-95..98
Closes the scope gap census 01 declared for itself: the seven instruments it named as uncensused. Pre-registered before any source or config was read, with predictions and a discrimination condition. Census 01 asked whether an instrument had a real negative instance — a question about CAPABILITY. Census 02 asks whether it has ever engaged in real life. Those come apart exactly at the drift-checker's shape, and 2026-08-04 found the gap six times (retrieval_count = 0 across 19,915 nodes for four months; two replay modules that have never processed an event). VERDICT: every instrument a human runs by hand has a rich firing record; every instrument that runs by itself has none — and the two guarding the engine's output have no consumer at all. The record divides by whether a human is in the invocation path, not by age, quality, or importance. verify-before-compose fired exactly twice (2026-07-17, 2026-07-18), evidence surviving only in harness transcripts; and it CANNOT fire on 31 of 59 guarded files, including the live constitution, because it folds the existing file's contents into its search for the attestation. audit_cruft, verify_conversion and apply_char_glyphs are exemplary. resolve_archived_source is healthy at 349/349 and has zero log entries. studium verify-quote and fidelity_equivalence@2 have no production call site at all. Prediction 5 inverted for the second census running, for a new reason. Census 01: decay, not construction, is the failure mode. Census 02: the recording is attached to the human, so an instrument's record vanishes the moment it is automated — which is when it starts running often enough to matter. Two of my own candidate findings died to their controls and are recorded as such: probing the resolver with engine source_ids against the chamber's canonical_slug key space (one sentence from "the resolver is inert"), and reading character_as_image at the wrong YAML nesting (nearly "zero glyph maps declared"; there are two sources and a 63-item census). Filed together: PENDING-95 [HARDENING] the hook cannot fire on the constitution · PENDING-96 [HARDENING] "SILENCE — ✓ warranted" certifies the index and claims the answer · PENDING-97 [PROPOSAL] FTS AND-s bare tokens with no semantic layer, recall dies as questions lengthen · PENDING-98 [HARDENING] firing history exists only where a human invokes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WuMjg3ipEVa3n8CoSzoyvc
This commit is contained in:
co-authored by
Claude Opus 5
parent
9c2a7c985c
commit
3a1790dcd6
@@ -0,0 +1,68 @@
|
||||
# Census 02 — Has each instrument ever produced a positive result?
|
||||
|
||||
**Pre-registered:** 2026-08-04, before reading any of the seven instruments' source or config.
|
||||
**Origin:** the literal question left by the 2026-08-04 wrap, closing the scope gap census 01 declared for itself.
|
||||
|
||||
---
|
||||
|
||||
## The question
|
||||
|
||||
Census 01 asked: *does each instrument have a real negative instance to test against, and is its absence recorded?* That is a question about the instrument's **capability**.
|
||||
|
||||
2026-08-04 found something one level more basic, six times over: an instrument can pass every check ever run against it while never having fired at all. `retrieval_count = 0` across 19,915 nodes for four months. `coherence_evaluated = 0` of 813,178 chains. `ANALYZE` never run in four months. Three of five `governance-drift-check.py` families inert. Two L1 replay modules that have never processed a single event — pinning the resume floor at zero and making every restart a full rebuild.
|
||||
|
||||
So, of each instrument: **has it ever produced a positive result — ever actually fired, matched, blocked, or caught something in real use — and does a record exist that could tell us either way?**
|
||||
|
||||
An instrument can be DEMONSTRATED at build time (census 01's gold standard) and inert ever since. That is not a hypothetical; it is precisely the drift-checker's shape.
|
||||
|
||||
## Unit of census
|
||||
|
||||
The seven instruments census 01 explicitly declared **out of scope**, censused entire:
|
||||
|
||||
1. `verify-before-compose` (the PreToolUse hook)
|
||||
2. `audit_cruft`
|
||||
3. `verify_conversion`
|
||||
4. `resolve_archived_source`
|
||||
5. `apply_char_glyphs`
|
||||
6. studium-engine `verify-quote` V1
|
||||
7. `fidelity_equivalence@2`
|
||||
|
||||
## The test applied to each
|
||||
|
||||
**Two independent axes, recorded separately.** Conflating them is the error this census exists to avoid — "it works" and "we can tell that it works" are different claims.
|
||||
|
||||
- **Axis A — engagement.** Is the instrument currently reachable? Does its trigger condition still match the substrate as the substrate now stands? Answerable from source and config.
|
||||
- **Axis B — firing record.** Does any durable record exist that would show a positive firing? Answerable by grep. A session transcript is not a durable record; a governance file, a log, a committed artefact is.
|
||||
|
||||
Four outcomes:
|
||||
|
||||
- **FIRED-RECORDED** — a durable record names a real occasion on which it fired.
|
||||
- **FIRED-UNRECORDED** — it demonstrably *can* fire and prose or memory attests that it did, but no durable record exists.
|
||||
- **INERT** — its trigger cannot currently match. It reports success by never engaging.
|
||||
- **UNDETERMINED** — cannot be settled on either axis. Reported as unclassifiable, not dropped.
|
||||
|
||||
## Predictions (recorded before the read)
|
||||
|
||||
1. **`verify-before-compose` is currently LIVE and has fired at least once.** ~0.75. Basis: `MEMORY.md` carries the standing preference *"Don't fight the block"* — which reads as a lesson earned by being blocked. That is prose attestation, not a record.
|
||||
2. **No durable firing record exists for `verify-before-compose`.** ~0.85. A PreToolUse hook blocks by exit code and stderr; the trace lands in a session transcript, which nothing reads.
|
||||
3. **At least one of the seven is INERT** — its trigger has drifted from the substrate it watches. ~0.6. Basis: census 01's prediction-5 inversion (decay, not construction, is the failure mode) compounded by six instances found on 2026-08-04.
|
||||
4. **The chamber-library tools (`audit_cruft`, `verify_conversion`, `apply_char_glyphs`) are FIRED-RECORDED**, via `_curation/tool-evolution-log.md`. Only ~0.5 — *the log exists* and *the log records firings* are different claims and I have checked neither.
|
||||
5. **Overall skew: FIRED-UNRECORDED dominant.** The instruments work; the record of their working does not exist.
|
||||
|
||||
## The census's own discrimination condition
|
||||
|
||||
A single instrument cannot discriminate anything. This census discriminates only if, across the seven, **both** a FIRED-RECORDED and an INERT (or FIRED-UNRECORDED) instance appear.
|
||||
|
||||
If all seven land in one bucket, the census has discriminated nothing and must be reported as such rather than as a result.
|
||||
|
||||
If the skew comes out opposite to prediction 5 — mostly FIRED-RECORDED — **the first hypothesis to test is that I accepted a weak record as a firing record**, not that our instruments are unusually well recorded.
|
||||
|
||||
## Falsifier for the central claim
|
||||
|
||||
For any instrument reported as having fired: **name the occasion and the artefact.** If I cannot name them, the verdict is FIRED-UNRECORDED at best. *"I remember it firing"* is not a record — it is the thing this census is trying to tell apart from a record.
|
||||
|
||||
## Stopping rule
|
||||
|
||||
Complete when all seven are classified on both axes. Unclassifiable members reported as unclassifiable.
|
||||
|
||||
**Order of execution:** `verify-before-compose` first — it is the only one of the seven that gates *steward-facing writes*, so silent inertness is worst there.
|
||||
@@ -0,0 +1,77 @@
|
||||
# Census 02 — RESULT
|
||||
|
||||
**Run:** 2026-08-04. Pre-registered at `census-02-have-they-ever-fired-PREREGISTRATION.md` before any instrument's source or config was read.
|
||||
**Question:** has each instrument ever produced a positive result — ever actually fired — and does a record exist that could tell us either way?
|
||||
|
||||
---
|
||||
|
||||
## Verdict in one line
|
||||
|
||||
**Every instrument that a human runs by hand has a rich firing record. Every instrument that runs by itself has none — and the two that guard the engine's output have no consumer at all.**
|
||||
|
||||
The census discriminated (both buckets populated, per the pre-registered condition), so it is reportable as a result rather than as nothing.
|
||||
|
||||
---
|
||||
|
||||
## The seven, censused entire
|
||||
|
||||
| # | Instrument | Axis A — engagement | Axis B — firing record | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| 1 | `verify-before-compose` | **LIVE**; blocks a new ungrounded artifact (exit 2, produced today). **Cannot fire on 31 of 59 guarded files**, including the live constitution | 2 firings — 2026-07-17 09:44:34 (Write), 2026-07-18 11:55:44 (Edit). Recoverable **only** from Claude Code session transcripts | **FIRED-UNRECORDED** |
|
||||
| 2 | `audit_cruft` | LIVE, wired, fleet-tested | Dense: 2026-06-16 `PASS-BUT-FALSELY` ×2 → **160 corpus files carrying residue the old gate was blind to** (95 image embeds, 64 raw svg, 8 `U+FFFD`); 2026-07-19 kindle-residue class | **FIRED-RECORDED** |
|
||||
| 3 | `verify_conversion` | LIVE, wired, fleet-tested | 2026-06-16 calibration — **948/952 pass on the known-clean Loeb tier, 4 genuine truncated stubs**; false-failed a clean Loeb Aeschylus → fixed; 2026-06-28 `PASS-BUT-FALSELY` in the gate itself | **FIRED-RECORDED** |
|
||||
| 4 | `resolve_archived_source` | **LIVE and healthy** — 349/349 manifest entries keyed; positive-controlled: real slugs resolve and the files exist on disk | **Zero mentions in `tool-evolution-log.md`.** It runs on every graduation and has never once been reviewed after use | **FIRED, UNLOGGED** |
|
||||
| 5 | `apply_char_glyphs` | LIVE, fleet-tested, REFUSE teeth intact; 2 sources mapped, 63-item census | 2026-07-23 — Levi, **527 docs, 0 unclassified, 0 residual glyph `<img>`**, recorded in both the log and the spec's declared data | **FIRED-RECORDED** |
|
||||
| 6 | studium `verify-quote` V1 | Functional — **42/42** deliberate-mismatch checks pass | **No production call site anywhere.** Only its own CLI `main()` and its test suite | **NO RECORD CAN TELL** |
|
||||
| 7 | `fidelity_equivalence@2` | Shipped and correct (`FIDELITY_VERSION = "fidelity_equivalence@2"`) | Imported by `verify_quote` alone — it fires exactly as often as #6 does, which is unrecorded | **NO RECORD CAN TELL** |
|
||||
|
||||
---
|
||||
|
||||
## The finding
|
||||
|
||||
**The tool-evolution log is the only mechanism in this system that successfully records firing history — and it only reaches tools a human runs by hand and then reviews.**
|
||||
|
||||
Where the discipline reaches (#2, #3, #5), the record is exemplary: dated, artifact-named, `PASS-BUT-FALSELY` recorded as the priority signal, tool patched and the reason logged. This is the strongest instrument record anywhere in the system, stronger than the verification ladder's and stronger than `fool/`'s.
|
||||
|
||||
Where it does not reach, nothing records:
|
||||
|
||||
- `verify-before-compose` fires inside the harness, so no human is present to log it. Its two firings survive only because Claude Code happens to retain transcripts — an accident of tooling with unknown retention, not a designed record.
|
||||
- `resolve_archived_source` runs inside `graduate_to_canonical` on every graduation. Because a human never *invokes* it, nobody ever *reviews* it. Zero log entries for a function that has run hundreds of times.
|
||||
- `verify-quote` and `fidelity_equivalence@2` are invoked by nothing at all.
|
||||
|
||||
The rule the log states — *"after **every** use — success or failure — we review how the tool actually behaved"* — is in practice **after every use a human initiates.** Automatic use is invisible to it by construction.
|
||||
|
||||
## Two structural findings, beyond the record
|
||||
|
||||
**1. `verify-before-compose` is disarmed on the artifact it exists to protect.** Lines 40–41 fold the *existing file's* contents into the search for the grounding attestation. Once a file contains `GROUNDED-IN:` anywhere, no future write to it can ever be gated. **31 of 59 guarded files are in that state**, including `chamber-library-specification.md`, every versioned spec v2.1.0–v2.9.0, and `graduation-spec.yaml`. The behaviour is documented ("or the existing file"; "a speed-bump… not a guarantee"); the *consequence* — coverage decaying monotonically to zero, and already zero on the constitution — is documented nowhere.
|
||||
|
||||
Countervailing evidence, recorded because it cuts the other way: **all 28 guarded files lacking a marker are dated ≤ 2026-07-17; every constitutional artifact created after the hook landed carries one.** Consistent with the gate having shaped behaviour even where it cannot now block. Filename dates are a proxy; creation dates are not git-verified.
|
||||
|
||||
**2. The engine's `SILENCE — ✓ warranted` certifies index currency and presents it as an answer about the corpus.** Asked `grey zone`, the engine replies *"No match — this is genuine silence, not a gap."* The corpus holds **ten** matches for `gray zone`, **all ten in `levi-drowned-and-saved`**. The warrant checks that the index is complete and current; it cannot check that retrieval reached what is there. This is verbatim the shape that opened census 01 — a check certifying a property of the code while claiming a property of the result — now at the engine's consuming end and wearing a checkmark.
|
||||
|
||||
Mechanism, and it is wider than spelling: `retrieve.py:103` passes the normalized user string straight to `drawers_fts MATCH`, where bare terms are **AND-ed**. Recall collapses as the question lengthens — `gray` 51 hits, `gray zone` 10, `levi the gray zone` **0**, `what does levi mean by the gray zone` **0**. Natural-language questions — precisely what *"enter into discourse with my library"* means — return certified silence by default. There is no vector table; `embed_spike.py` / `rerank_spike.py` remained spikes.
|
||||
|
||||
Discrimination held here too: the probe *"the quality without a name"* is **correctly** silent — *The Timeless Way of Building* is not among the 13 sources. One real defect and one real correct silence, side by side.
|
||||
|
||||
## Residual findings
|
||||
|
||||
- **No `.glyph-manifest.json` has ever been written anywhere.** The two-sided manifest (REVIEWED-74) is the input to `resolve_added`'s added-side fabrication check; its wiring is exercised only by a synthetic fixture in `test_tools.py`. The check has never seen a real manifest.
|
||||
- **The engine's `source_id` and the chamber's `canonical_slug` are disjoint key spaces for the same works** (`levi-drowned-and-saved` vs `the-drowned-and-the-saved-primo-levi`), with nothing joining them. A 7th instance of the shared-name/two-senses class the chamber CLAUDE.md counts.
|
||||
- `engine/verify_quote.py:7` names `fidelity_equivalence@1` in prose while the shipped constant is `@2`. Stale docstring; `[FIX]` class.
|
||||
|
||||
## Two of my own claims died to their controls — recorded, because that is the census working
|
||||
|
||||
- I probed `resolve_archived_source` with engine `source_id`s and got `None` three times, including for the nonsense control. Reporting "the resolver is inert" was one sentence away. The positive control — resolving real slugs *drawn from the manifest itself* — showed it healthy at 349/349. **The instrument was mine, and it was wrong.**
|
||||
- I read `character_as_image` at the YAML top level, got nothing, and nearly reported "zero glyph maps declared." It is nested at `promotion.character_as_image` and holds **2 sources and a 63-item census**.
|
||||
|
||||
Both are the day's own failure class (a number from an uncontrolled instrument), and both died to a control run before the claim left the workspace.
|
||||
|
||||
## Predictions, graded
|
||||
|
||||
1. `verify-before-compose` LIVE and fired ≥ once (~0.75). **Held** — exactly twice, both dated.
|
||||
2. No durable firing record for it (~0.85). **Held.**
|
||||
3. At least one of the seven INERT (~0.6). **Reframed, not held.** None is inert in the drift sense — every one still engages what it was built to engage. The real shapes were *disarmed-by-accumulation* (#1) and *no-consumer* (#6, #7), neither of which I anticipated.
|
||||
4. Chamber tools FIRED-RECORDED via the log (~0.5). **Held and stronger than predicted** for #2/#3/#5 — and **wrong for #4**, which is a chamber tool with zero log entries because no human invokes it.
|
||||
5. Overall skew FIRED-UNRECORDED dominant. **Wrong.** The skew is not by age or by quality; it is by **whether a human is in the invocation path**.
|
||||
|
||||
Prediction 5 has now inverted twice, in two censuses, for two different reasons. Census 01: decay, not construction, is the failure mode. Census 02: **the recording discipline is attached to the human, not to the instrument** — so an instrument's record vanishes the moment it is automated, which is exactly when it starts running often enough to matter.
|
||||
@@ -70,3 +70,22 @@ metadata:
|
||||
## Sub-agent dialogues
|
||||
|
||||
## Bypasses
|
||||
|
||||
---
|
||||
|
||||
# Second sitting — 2026-08-04 (post-clear)
|
||||
|
||||
## Returns
|
||||
|
||||
- **2026-08-04T13:40 CEST — corrected the inherited resumption path rather than following it.** The wrap named `chamber-library/_curation/fool/census-01-negative-instances-RESULT.md`; that directory does not exist. The census lives at `~/dotfiles/claude/governance/fool/` (RESULT + PREREGISTRATION). Found by `find`, not assumed. A wrap-authored path is a claim like any other; the cheapness of the check is the point.
|
||||
|
||||
## Confidence to recalibrate
|
||||
|
||||
- **Standing, inherited and unchanged:** *a number that matches a constant is not a cause* — four refutations yesterday, all resting on arithmetic agreement with a code constant. And *knowing the name of a failure class confers no immunity to it*. Both bear directly on today's thread, which is precisely about instruments that report success without ever having engaged.
|
||||
- **Not yet substrate-checked:** the 13 open PENDING items were carried from the wake digest, not re-verified against their own disposition. Three were filed today (92/93/94) and are verified by authorship. The other ten are **unverified — last confirmed at filing**.
|
||||
|
||||
## Open horizons
|
||||
|
||||
- **PULLING THREAD (inherited, confirmed):** census 01 against the seven uncovered chamber/engine gates — `verify-before-compose` · `audit_cruft` · `verify_conversion` · `resolve_archived_source` · `apply_char_glyphs` · studium `verify-quote` · `fidelity_equivalence@2`. Question: has each ever produced a positive result, and does a record exist that could tell us either way?
|
||||
- Parked hard: the 2.32 events/min rate (needs a controlled measurement, four theories lost).
|
||||
- Awaiting steward: PENDING-92 annotation before its local 3-commit branch moves.
|
||||
|
||||
Reference in New Issue
Block a user