Skip to content

refactor(skills): retire the context-manager scripts and align every Skill name with its directory - #17

Merged
jiunbae merged 4 commits into
mainfrom
chore/demote-context-manager
Aug 28, 2026
Merged

jiunbae merged 4 commits into
mainfrom
chore/demote-context-manager

Conversation

@jiunbae

@jiunbae jiunbae commented Aug 28, 2026 •

Copy link
Copy Markdown
Owner

Two commits. The first was the original scope; the second closes a naming split
the first one surfaced.


1 — context-manager loses its scripts and leaves core (e75ecc2)

find_context.py presented itself as a weighted relevance ranker. Measured
against a real context/ directory:

$ find_context.py --context-dir IaC/context --keywords deploy onboarding \
    --task-type implementation
1. context/README.md                              0.700   ← pinned, always first
2. context/operations/service-onboarding.md       0.209   ← the correct answer
3. context/architecture/sealed-secrets.md         0.204
4. context/architecture/security-audit-report.md  0.204
5. context/architecture/nfs-storage.md            0.200   ← unrelated

The correct document beat an unrelated one by 0.009. The causes are structural:

  • Bodies are never read. score_keyword_match looks at the filename stem
    and the parent directory name only, so any term that lives in the text —
    the normal case — scores zero.
  • README.md is hardcoded to 0.7, so it always ranks first.
  • The 10% recency weight is inverted in effect: a document untouched for
    90 days scores zero, penalising the stable, canonical procedures.
  • score_path_match compares code path segments against context category
    names; the vocabularies rarely intersect.
  • min_score=0.1 is met by recency alone.

grep -ril over document bodies is strictly more accurate, so SKILL.md now
says that. update_context.py goes too — the skill does not write documents on
its own initiative, and its mkdir contradicted the instruction not to create
the directory. references/context_patterns.md (524 lines) duplicated
static/CONTEXT.md and cited layouts that no longer exist.

The core profile is defined in five places that had to stay in agreement:
profiles.yml, install.sh (array + help text), install.ps1, README.md,
README_ko.md. The skill stays in full and installable by path.

2 — every Skill declares the name a reader types (30d992a)

e529d78 renamed 22 Skills' name frontmatter to gerund phrases. A host lists
a Skill by its directory, so managing-vault-secrets on
integrations/vault-secrets was a second name nobody could type.

Not cosmetic: air-workbench's catalog keys items on the frontmatter name, so the
catalog and the host disagreed about what a Skill is called — catalog.test.mjs
asserted analyzing-business-model for a directory named bm-analyzer.

Renaming the directories was the alternative and is worse: it breaks every
published --from jiunbae/agent-skills/<group>/<skill> path and the CHANGELOG's
own history. So the declaration moved, in 22 one-line edits.

tests/core.test.mjs gains a guard that fails with the offending path on any
future divergence — verified by reintroducing a mismatch and watching it fail.
The rule is documented in both READMEs.

Test fallout, explained

  • Frozen security-auditor node ids changed. A node id hashes the heading's
    byte offset; the frontmatter line is one byte shorter. Titles and graph shape
    are unchanged — recomputed and updated.
  • The playwright search fixtures are synthetic and keep testing a divergent
    name, which an author can still write. Only their comments changed: they no
    longer claim the repository contains such a Skill.

Verification

  • bash -n install.sh; --help lists the 7 remaining core skills.
  • air-workbench: 355 tests pass (354 before, +1 guard).
  • Drift guard negative-tested.
  • No dangling references to removed files or retired names.

🤖 Generated with Claude Code

jiunbae and others added 2 commits August 28, 2026 17:47
…e profile

`find_context.py` claimed to rank context documents by relevance but scored
only filenames and parent directory names, never document bodies — so a term
that appears in the text, which is the normal case, scored zero. `README.md`
was pinned to 0.7 and therefore always ranked first, and the 10% recency weight
penalised exactly the stable, canonical documents most worth reading. Run
against a real `context/` directory with matching keywords, the gap between the
correct document and an unrelated one was 0.009. That is noise, not a ranking.

`grep -ril` over document bodies is strictly more accurate, so SKILL.md now
says to do that and the script is gone. `update_context.py` goes with it: the
skill does not write documents on its own initiative, and the script's
`mkdir` contradicted the instruction not to create the directory.
`references/context_patterns.md` duplicated `static/CONTEXT.md` and cited
project layouts that no longer exist.

The skill leaves the `core` profile, which is defined in five places that had
to agree: profiles.yml, install.sh (array and help text), install.ps1, and both
READMEs. It stays installable on request and keeps its `managing-context`
frontmatter name, the gerund convention e529d78 applied across the catalog.

air-workbench's recognized-shape fixture is unchanged: the rewritten SKILL.md
still imports as 4 nodes / 3 edges under `workflow.children`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`e529d78` renamed 22 Skills' `name` frontmatter to gerund phrases —
`managing-vault-secrets` on `integrations/vault-secrets`, `auditing-security` on
`security/security-auditor` — citing the official guidance on naming a Skill.
The guidance is about choosing the Skill's name; it does not ask the declaration
to disagree with the directory, and here it does.

A host lists a Skill by its directory. `managing-vault-secrets` was therefore a
second name nobody could type or invoke. It was not merely cosmetic either:
air-workbench's catalog keys items on the frontmatter name, so the catalog and
the host disagreed about what a Skill is called, and `catalog.test.mjs` had to
assert `analyzing-business-model` for a directory named `bm-analyzer`.

Renaming the directories to match the gerund names was the other way to close
the gap, and it is worse: every published
`--from jiunbae/agent-skills/<group>/<skill>` install path would break, along
with the CHANGELOG's own history. The declaration moves instead.

`tests/core.test.mjs` gains a guard that fails with the offending path on any
future divergence; it was verified to fail by reintroducing one. The frozen
`security-auditor` node ids move because a node id hashes the heading's byte
offset and the frontmatter line is one byte shorter — titles and shape are
unchanged. The playwright-based search fixtures keep testing a divergent name,
which a Skill author can still write; their comments no longer claim the
repository contains one.

354 → 355 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jiunbae jiunbae changed the title refactor(context-manager): drop the scoring scripts and leave the core profile refactor(skills): retire the context-manager scripts and align every Skill name with its directory Aug 28, 2026
jiunbae and others added 2 commits August 28, 2026 18:28
The pointer added with the demotion omitted `-g`, so following it installs the
skill into the current project instead of the user's global skill directory —
the opposite of what a workspace-wide opt-in needs. Match the `agt` form the
README already documents for `agents/air-workbench`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…inventory

The release gate pins a per-file TAP plan so a suite cannot lose a test
silently. Adding the Skill-naming guard moved core.test.mjs from 46 to 47, and
the gate correctly refused the branch until the inventory agreed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jiunbae
jiunbae merged commit 51377ac into main Aug 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant