Skip to content

Commit b65ae15

Browse files
neuromechanistneuromechanist
committed
Merge feature/issue-137-official-hed-docs into develop
Replace hand-written HED rules with official docs (#137) Closes #137, closes #69, closes #100
2 parents 6f448d8 + 2cbcd42 commit b65ae15

18 files changed

Lines changed: 2502 additions & 973 deletions

‎.context/hed-annotation-rules.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
# HED Annotation Rules
22

33
## Source
4-
These rules are derived from the HED specification and implemented in `src/utils/hed_rules.py`. The complete system prompt generation is in `get_complete_system_prompt()`.
4+
The annotation agent system prompt now uses official HED documentation
5+
pulled from the hed-standard GitHub repos (HedAnnotationSemantics.md
6+
and 02_Terminology.md), bundled in `src/data/hed-docs/` and loaded by
7+
`src/utils/hed_docs_loader.py`. HEDit-specific sections (vocabulary
8+
check, correction workflow, error troubleshooting, output format)
9+
are in `src/utils/hed_comprehensive_guide.py`.
510

611
## Fundamental Principles
712

‎.context/hed-schemas.md‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# HED Schema Structure and Access
22

33
## Current Version
4-
- **Standard schema**: HED 8.3.0 (HEDit default), 8.4.0 (OSA/hedtools.org default)
4+
- **Standard schema**: HED 8.4.0 (HEDit default)
55
- **Library schemas**: HED_score, HED_lang, etc.
66

77
## Schema Loaders
@@ -51,5 +51,6 @@ The JSON schema loader extracts:
5151

5252
## Usage in Agents
5353
- Annotation agent receives full vocabulary + extendable tags in system prompt
54-
- Only first 80 vocabulary tags shown as sample (full list would exceed context)
55-
- ExtensionAllowed tags shown (first 20) so agent knows what can be extended
54+
- Official HED docs (HedAnnotationSemantics.md, 02_Terminology.md) are preloaded
55+
from bundled files in src/data/hed-docs/ (fetched by scripts/fetch_hed_docs.py)
56+
- ExtensionAllowed tags shown so agent knows what can be extended

‎.context/osa-comparison-analysis.md‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11
# HEDit vs OSA/HBD Comparison Analysis
22

3-
## Summary
3+
## Status
4+
As of issue #137, the documentation gap has been closed. HEDit now preloads
5+
official HED docs (HedAnnotationSemantics.md, 02_Terminology.md) from bundled
6+
files, matching OSA's approach. A weekly CI workflow keeps the docs current.
7+
8+
## Original Summary (historical)
49
The Open Science Assistant (OSA) HED assistant produces more thorough and accurate responses than HEDit. This analysis identifies the key differences and recommends improvements.
510

611
## Key Differences

‎.github/workflows/update-hed-docs.yml‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Update HED Docs
2+
3+
on:
4+
schedule:
5+
- cron: '0 6 * * 1' # Weekly Monday at 6am UTC
6+
workflow_dispatch: # Manual trigger
7+
8+
permissions:
9+
contents: write
10+
pull-requests: write
11+
12+
jobs:
13+
update-docs:
14+
name: Fetch Latest HED Docs
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v6
18+
with:
19+
ref: develop
20+
token: ${{ secrets.RELEASE_PAT }}
21+
22+
- uses: actions/setup-python@v6
23+
with:
24+
python-version: '3.12'
25+
26+
- uses: astral-sh/setup-uv@v7
27+
28+
- name: Install dependencies
29+
run: uv pip install --system httpx
30+
31+
- name: Fetch latest HED docs
32+
run: python scripts/fetch_hed_docs.py
33+
34+
- name: Check for changes
35+
id: changes
36+
run: |
37+
if git diff --quiet src/data/hed-docs/; then
38+
echo "changed=false" >> $GITHUB_OUTPUT
39+
else
40+
echo "changed=true" >> $GITHUB_OUTPUT
41+
fi
42+
43+
- name: Create PR if docs changed
44+
if: steps.changes.outputs.changed == 'true'
45+
run: |
46+
git config user.name "github-actions[bot]"
47+
git config user.email "github-actions[bot]@users.noreply.github.com"
48+
BRANCH="auto/update-hed-docs-$(date +%Y%m%d)"
49+
git checkout -b "$BRANCH"
50+
git add src/data/hed-docs/
51+
git commit -m "Update bundled HED docs from upstream
52+
53+
Automated update from official HED documentation.
54+
55+
[skip-release]"
56+
git push origin "$BRANCH"
57+
gh pr create \
58+
--base develop \
59+
--title "Update bundled HED documentation" \
60+
--body "$(cat <<'BODY'
61+
## Summary
62+
Automated weekly update of bundled HED documentation from upstream sources.
63+
64+
Sources:
65+
- hed-standard/hed-resources (HedAnnotationSemantics.md)
66+
- hed-standard/hed-specification (02_Terminology.md)
67+
68+
This PR was created by the update-hed-docs workflow.
69+
BODY
70+
)" \
71+
--label "documentation"
72+
env:
73+
GH_TOKEN: ${{ secrets.RELEASE_PAT }}

‎pyproject.toml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
44

55
[project]
66
name = "hedit"
7-
version = "0.7.6.dev5"
7+
version = "0.7.7.dev0"
88
description = "Multi-agent system for HED annotation generation and validation"
99
readme = "PKG_README.md"
1010
requires-python = ">=3.12"
@@ -152,6 +152,9 @@ exclude_lines = [
152152
where = ["."]
153153
include = ["src*"]
154154

155+
[tool.setuptools.package-data]
156+
"src.data" = ["hed-docs/*.md", "hed-docs/*.json"]
157+
155158
[tool.mypy]
156159
python_version = "3.12"
157160
warn_return_any = true

0 commit comments

Comments
 (0)