2.2 KiB
name, description, metadata
| name | description | metadata | ||||||
|---|---|---|---|---|---|---|---|---|
| feedback-tool-review-after-each-use | Steward directive — review how our tools behaved after EVERY use (success or failure), capture what it taught, iterate until reliable. The honest-state loop turned on our own toolmaking. |
|
After any run of a tool we've built (conversion, audit, repair, census, the engine's own scripts), review how it actually behaved before moving on — on success as much as on failure — and capture what it taught, then iterate. The goal is tools that do not misrepresent their own state: report clean only when clean, succeed only when they succeeded, fail loud when they did not. Iterate until absolutely reliable; a tool is never "finished."
Why: Steward directive, 2026-06-16. Source quality determines retrieval quality, and the chamber's whole reason for being is to refuse state-misrepresentation. A tool that says "clean" on a dirty file or "done" on a broken result is the same contamination shape as a confident-but-wrong answer. The standing review loop is how the tools earn trust — proven the same day: hardening audit_cruft surfaced 160 falsely-"clean" files the old gate hid; testing the new verifier caught its own Greek false-positive; the repair fix went 4→12 headings. PASS-BUT-FALSELY (tool reports success on a wrong result) is the priority signal — it earns an immediate patch or a logged proposal, never a shrug.
How to apply:
- For chamber-library tools, append a review entry to
chamber-library/_curation/tool-evolution-log.md(date · tool · source · outcome · what surfaced · improvement applied/proposed · follow-up). The protocol lives at the top of that file. - A new failure shape → a new check/pattern in the tool, recorded in the log AND the tool's header.
- "No improvement needed" is a valid, useful outcome — record it; absence of a finding is information.
- Generalizes to the Studium Engine scripts and any tool we build. Operationalizes
conversion-skill-plan-2026-05-17.md§13 (verification surface is a moving target). Kin to reference-verification-ladder and the skill-harvest-register.