[Feature] Add skill retrieval to load only relevant skills instead of injecting all skills into context

[Y0uYuGe]

Summary

Add a skill retrieval mechanism so that, when many skills are available, the system can retrieve and load only the most relevant skills for the current task instead of injecting all skill content into the model context.

Motivation or Use Case

As the number of available skills grows, loading every skill into the model context becomes increasingly inefficient and harmful.

Current pain points:

• context usage grows quickly when many skills are installed
• irrelevant skills introduce noise and reduce instruction clarity
• token cost increases unnecessarily
• model attention is diluted across unrelated skills
• runtime quality may degrade because the agent sees too many possible capabilities at once

This becomes especially problematic in multi-agent or shared-skill setups, where a user may have access to many reusable skills, but only a small subset is relevant to a given task.

Who benefits:

• users with large shared skill libraries
• agent applications that want lower context cost
• frameworks that need scalable skill loading
• teams managing many domain-specific or internal skills

Proposed Change

Introduce a retrieval-based skill selection workflow before skill injection.

1. Skill indexing

Allow the system to build an index over installed or available skills using information such as:

• skill name
• description
• trigger phrases
• tags or categories
• SKILL.md metadata
• examples or usage hints
• referenced commands or tools

This index can be keyword-based, embedding-based, or hybrid.

2. Retrieval before loading

Before injecting skill content into the model context, the system should:

• analyze the current user task
• retrieve the most relevant skills
• load only the top-k matching skills or their summaries
• optionally expand to full skill content only when needed

3. Multi-stage loading

To reduce context usage further, support staged loading:

• stage 1: retrieve lightweight skill metadata or summaries
• stage 2: select likely relevant skills
• stage 3: load the full content of only the chosen skills

This avoids injecting every skill in full form up front.

4. Configurable retrieval behavior

Possible controls:

• top-k skill limit
• retrieval threshold
• keyword / semantic / hybrid search mode
• per-agent skill scope
• fallback behavior when no skill matches well

5. Optional explanation

It would be helpful if the system can show:

• which skills were retrieved
• why they were selected
• which skills were ignored due to low relevance

This would improve debuggability and trust.

6. Compatibility with existing skill workflows

This feature should work with current skill packaging and loading flows.
The goal is not to change the skill format, but to add a scalable retrieval layer in front of context injection.

Alternatives Considered

Alternative 1: Inject all skills into context

Simple, but does not scale.
This increases token cost, adds noise, and becomes impractical when many skills are installed.

Alternative 2: Manually pre-select skills per agent

This helps reduce scope, but still requires users to predict in advance which skills will be needed.
It also does not solve task-level dynamic selection.

Alternative 3: Hard-coded trigger rules

This may work for a small number of skills, but becomes brittle and difficult to maintain as the skill set grows.

Preferred direction

A retrieval-based approach is more scalable because it allows the system to choose skills dynamically based on the actual task, while keeping context usage under control.

Documentation or Example Impact

The following documentation would likely need updates:

• skill loading behavior
• agent configuration docs
• guidance for writing better skill metadata for retrieval
• examples showing retrieval-based skill selection
• notes on how summaries / tags / triggers affect matching quality

It may also be useful to add examples of:

• a small skill library
• a large skill library
• how the same task retrieves different skills depending on context

Additional Context

This feature is especially important for any skill ecosystem that aims to support:

• many reusable skills
• shared skill repositories
• per-agent selective exposure
• scalable cross-runtime interoperability

Without retrieval, skill reuse does not scale well because every additional skill increases context overhead.

With retrieval, the system can treat skills more like a searchable capability library:

• keep a large skill pool available
• retrieve what is relevant for the current task
• load only what is needed

A future extension could include:

• learned ranking for skill retrieval
• retrieval logs for debugging
• agent-specific retrieval policies
• integration with skill security scanning
• retrieval-aware skill summaries optimized for context efficiency