Summary
RLCR's BitLesson knowledge base (.humanize/bitlesson.md) is effectively append-only.
The round stop-gate validates the format of the ## BitLesson Delta (per docs/bitlesson.md:
Action ∈ none|add|update, IDs must exist), but nothing validates that existing lessons
are still accurate against the current repo. As the codebase refactors, lessons rot —
and a rotted lesson handed to an implementer by bitlesson-select.sh is worse than no
lesson, because it actively misleads.
Where it breaks (observed in a real knowledge base after one dir reorg)
A subsystem's code moved eval_audio_cot/scripts/ → scripts/audio_cot/ (output
namespace stayed outputs/eval_audio_cot/). All of the following passed the format-only gate:
- 3 lessons'
Scope: lines still point at the old eval_audio_cot/scripts path (dir gone).
- A
Solution: snippet says derive root via Path(__file__).resolve().parent.parent,
but the file is now nested one level deeper, so current code uses parent.parent.parent
— copying the lesson literally is off-by-one.
- A canonical path
outputs/analysis/{score,go_nogo_decision}.json moved to
outputs/eval_audio_cot/analysis/....
Update (round 9) notes superseded by Update (round 22) linger with no way to retire them.
Root cause
- The Delta contract has no
deprecate/delete action (only none|add|update), so
superseded lessons and stale Update (round N) notes can only be piled on, never retired.
- No step ever re-checks that a lesson's cited paths /
file:line refs / code snippets
still resolve. (Cf. the agent-memory rule: "a memory naming a file is a claim it existed
when written — verify before acting." BitLessons make the same claims but are never re-verified.)
Proposed improvements
- Add
deprecate (and/or delete) to the BitLesson Delta (Action: none|add|update|deprecate),
with provenance ("superseded by BL-… / dir reorg"). Lets the loop retire dead lessons
and consolidate old Updates.
- A reference-resolution / staleness check — companion to
bitlesson-select.sh, run at
loop start or as a periodic gate — that parses each lesson's Scope paths + file:line
refs and reports entries whose references no longer exist. Advisory by default; optionally
blocking via a flag (like the existing --require-bitlesson-entry-for-none).
- Selector surfaces staleness at point of use: when a lesson is selected, re-validate its
referenced paths; if any don't resolve, tag it "possibly stale — verify before applying".
- Update-note curation: when an entry accrues many
Update (round N) lines, prompt to fold
them into the Solution and drop superseded ones.
Scope / non-goals
Lesson content (the bug→fix knowledge) usually stays valid across refactors; only its
references drift. Default should be fix/flag references, not delete whole lessons —
deletion reserved for genuinely obsolete subsystems via the explicit deprecate action.
Summary
RLCR's BitLesson knowledge base (
.humanize/bitlesson.md) is effectively append-only.The round stop-gate validates the format of the
## BitLesson Delta(perdocs/bitlesson.md:Action ∈ none|add|update, IDs must exist), but nothing validates that existing lessons
are still accurate against the current repo. As the codebase refactors, lessons rot —
and a rotted lesson handed to an implementer by
bitlesson-select.shis worse than nolesson, because it actively misleads.
Where it breaks (observed in a real knowledge base after one dir reorg)
A subsystem's code moved
eval_audio_cot/scripts/→scripts/audio_cot/(outputnamespace stayed
outputs/eval_audio_cot/). All of the following passed the format-only gate:Scope:lines still point at the oldeval_audio_cot/scriptspath (dir gone).Solution:snippet says derive root viaPath(__file__).resolve().parent.parent,but the file is now nested one level deeper, so current code uses
parent.parent.parent— copying the lesson literally is off-by-one.
outputs/analysis/{score,go_nogo_decision}.jsonmoved tooutputs/eval_audio_cot/analysis/....Update (round 9)notes superseded byUpdate (round 22)linger with no way to retire them.Root cause
deprecate/deleteaction (only none|add|update), sosuperseded lessons and stale
Update (round N)notes can only be piled on, never retired.file:linerefs / code snippetsstill resolve. (Cf. the agent-memory rule: "a memory naming a file is a claim it existed
when written — verify before acting." BitLessons make the same claims but are never re-verified.)
Proposed improvements
deprecate(and/ordelete) to the BitLesson Delta (Action: none|add|update|deprecate),with provenance ("superseded by BL-… / dir reorg"). Lets the loop retire dead lessons
and consolidate old Updates.
bitlesson-select.sh, run atloop start or as a periodic gate — that parses each lesson's Scope paths +
file:linerefs and reports entries whose references no longer exist. Advisory by default; optionally
blocking via a flag (like the existing
--require-bitlesson-entry-for-none).referenced paths; if any don't resolve, tag it "possibly stale — verify before applying".
Update (round N)lines, prompt to foldthem into the Solution and drop superseded ones.
Scope / non-goals
Lesson content (the bug→fix knowledge) usually stays valid across refactors; only its
references drift. Default should be fix/flag references, not delete whole lessons —
deletion reserved for genuinely obsolete subsystems via the explicit
deprecateaction.