Merge pull request #164 from Annotation-Garden/feature/phase2-universal-caching-observability

Complete Phase 2: prompt-cache measurement, usage reporting, and migration cleanup

Author: neuromechanist

.context/agent-architecture.md

@@ -68,7 +68,7 @@ HEDit uses a LangGraph-based multi-agent workflow to convert natural language ev
 
 ### 6. VisionAgent (`src/agents/vision_agent.py`)
 - **Input**: Base64-encoded image
-- **Process**: Uses vision-language models (e.g., Qwen-VL via OpenRouter)
+- **Process**: Uses a Claude model (natively multimodal; no separate vision vendor)
 - **Output**: Natural language description of image content
 - **Key feature**: Enables image-to-HED annotation pipeline
 
@@ -100,7 +100,12 @@ Defined in `src/agents/state.py` as `HedAnnotationState(TypedDict)`:
 
 ## Configuration
 
-- **LLM Provider**: OpenRouter API (production), with Ollama fallback
+- **LLM Provider**: Anthropic Claude via the Claude Platform on AWS (Anthropic-operated
+  Messages API, AWS Marketplace billing; not Amazon Bedrock)
+- **Offered models**: `claude-haiku-4-5` (default) and `claude-sonnet-5`
 - **Default model**: Configurable via environment/headers
-- **BYOK support**: Users can provide their own API keys
+- **BYOK support**: Users can provide their own Anthropic key (`X-Anthropic-Key`),
+  which routes to api.anthropic.com
+- **Prompt caching**: The annotation system prompt is cached; usage and savings are
+  reported per request (see `docs/prompt-caching.md`)
 - **Streaming**: LangGraph `astream_events` for real-time progress via SSE

.context/api-and-deployment.md

@@ -11,17 +11,23 @@
 - `POST /feedback`: User feedback submission
 - `GET /health`: Service health check
 - `GET /version`: Version information
+- `GET /metrics`: Token use, cost, and prompt-cache savings since startup
+  (server API key required; BYOK callers get 403)
 
 ### Authentication Modes
-1. **Server mode**: `X-API-Key` header (server's OpenRouter key)
-2. **BYOK mode**: `X-OpenRouter-Key` header (user's own API key)
+1. **Server mode**: `X-API-Key` header (server's Anthropic credentials are used)
+2. **BYOK mode**: `X-Anthropic-Key` header (user's own Anthropic key, `sk-ant-...`,
+   routed to api.anthropic.com)
 3. **Public endpoints**: `/feedback`, `/health`, `/version`
 
 ### Model Override Headers
-- `X-OpenRouter-Model`: Override annotation model
-- `X-OpenRouter-Vision-Model`: Override vision model
-- `X-OpenRouter-Provider`: Override provider routing
-- `X-OpenRouter-Temperature`: Override temperature
+- `X-Anthropic-Model`: Override annotation model
+- `X-Anthropic-Eval-Model`: Override evaluation model
+- `X-Anthropic-Vision-Model`: Override vision model
+- `X-Anthropic-Temperature`: Override temperature
+
+The legacy `X-OpenRouter-*` spellings of these names (plus `X-OpenRouter-Key`) are
+still accepted as transport; provider-routing headers are ignored.
 
 ### CORS
 - Production: `hedit.pages.dev`, `annotation.garden`

.env.example

@@ -74,6 +74,18 @@ VISION_MODEL=claude-haiku-4-5
 # For highest annotation quality (slower, more expensive):
 # ANNOTATION_MODEL=claude-sonnet-5
 
+# ============================================================================
+# Prompt Cache Lifetime
+# ============================================================================
+# The annotation system prompt (~21.8k tokens of HED vocabulary guide) is
+# cached, which cuts input cost by ~90% on a cache hit. Entries live 5
+# minutes by default, which suits server traffic; a cache write costs 1.25x,
+# so break-even is the second request inside the window.
+# Set to 1h for interactive use where requests are minutes apart (a 1-hour
+# write costs 2x and pays off from the third request in the hour).
+# See docs/prompt-caching.md
+# HEDIT_PROMPT_CACHE_TTL=5m
+
 # ============================================================================
 # HED Configuration
 # ============================================================================

.rules/testing.md

@@ -14,7 +14,8 @@ Never use mocks, stubs, or fake datasets. If real testing is not possible, don't
 
 ## When to Write Tests
 - **DO:** Test with real data and actual dependencies
-- **DO:** Use test API keys (`OPENROUTER_API_KEY_FOR_TESTING`)
+- **DO:** Use the Anthropic test credentials (`ANTHROPIC_API_KEY`, with
+  `ANTHROPIC_BASE_URL` and `ANTHROPIC_WORKSPACE_ID` for the Claude Platform on AWS)
 - **DO:** Test against actual file systems and real HED schemas
 - **DON'T:** Write tests if only mocks would work
 - **DON'T:** Create artificial test scenarios

.serena/memories/codebase_structure.md

@@ -22,7 +22,8 @@ hedit/
 │   │   ├── json_schema_loader.py  # JSON schema + vocabulary extraction
 │   │   ├── schema_loader.py       # Legacy Python schema loader
 │   │   ├── error_remediation.py   # Error augmentation for LLM feedback
-│   │   ├── openrouter_llm.py      # OpenRouter API integration
+│   │   ├── anthropic_llm.py       # Anthropic Claude integration (prompt caching)
+│   │   ├── llm_usage.py           # Token, cost, and cache accounting
 │   │   ├── litellm_llm.py         # Alternative LLM providers
 │   │   ├── image_processing.py    # Base64 image encoding
 │   │   └── github_client.py       # GitHub API for feedback issues

.serena/memories/project_overview.md

@@ -7,7 +7,7 @@ HEDit is a multi-agent system for converting natural language event descriptions
 - **Language**: Python 3.12+
 - **Package Manager**: uv
 - **Agent Framework**: LangGraph
-- **LLM Provider**: OpenRouter API (production), Ollama (fallback)
+- **LLM Provider**: Anthropic Claude via the Claude Platform on AWS (claude-haiku-4-5 default, claude-sonnet-5 optional)
 - **Validation**: HED JavaScript validator + HED Python tools (hedtools)
 - **Backend**: FastAPI
 - **CLI**: Typer + Rich

README.md

@@ -67,6 +67,7 @@ Config files are stored in `~/.config/hedit/`:
 - **Image Annotation**: Annotate visual stimuli directly from image files
 - **Multi-Stage Validation**: AI agents generate, validate, evaluate, and refine annotations
 - **Claude-Powered**: Anthropic Claude models (Haiku 4.5 default, Sonnet 5 optional); bring your own Anthropic key if you prefer your own billing
+- **Cost Transparency**: Every annotation reports its token use and how much prompt caching saved (typically ~80% of input cost after the first request)
 - **JSON Output**: Easy integration with scripts and pipelines
 - **HED Schema Support**: Works with official HED schemas (8.x)
 
@@ -83,6 +84,7 @@ The agents work in feedback loops, automatically refining the annotation until i
 
 ## Documentation
 
+- [Prompt Caching and Usage Reporting](docs/prompt-caching.md) - What HEDit caches, what it saves, and where to see the numbers
 - [HED Standard](https://hedtags.org) - Learn about HED annotations
 - [GitHub Issues](https://github.com/Annotation-Garden/HEDit/issues) - Report bugs or request features
 
@@ -234,6 +236,7 @@ uvicorn src.api.main:app --reload --host 0.0.0.0 --port 38427
 - `POST /annotate`: Generate HED annotation from natural language
 - `POST /validate`: Validate HED annotation
 - `GET /health`: Health check
+- `GET /metrics`: Token use, cost, and prompt-cache savings since startup (server API key required)
 - API URL: `http://localhost:38427`
 
 ## Development

SECURITY.md

@@ -123,9 +123,11 @@ See [`deploy/SECURITY.md`](deploy/SECURITY.md) for complete security documentati
 ## Known Security Considerations
 
 ### LLM API Keys
-- OpenRouter API keys are stored as environment variables
-- Keys are passed to OpenRouter API via HTTPS
+- Anthropic API keys are stored as environment variables (`ANTHROPIC_API_KEY`,
+  plus `ANTHROPIC_BASE_URL` and `ANTHROPIC_WORKSPACE_ID` for the Claude Platform on AWS)
+- Keys are sent to the Anthropic Messages API over HTTPS
 - Keys are not logged in audit logs
+- BYOK keys supplied via `X-Anthropic-Key` are used for that request only, never stored
 - Rotate keys if compromised
 
 ### Cloudflare Worker Proxy
@@ -134,11 +136,11 @@ See [`deploy/SECURITY.md`](deploy/SECURITY.md) for complete security documentati
 - Not visible in frontend JavaScript
 - Recommended for production deployments
 
-### Local GPU Deployment
-- Ollama runs locally (no API key needed)
-- No external API calls for LLM inference
-- Complete privacy for offline operation
-- Ensure GPU drivers are up to date
+### Data Sent to the LLM
+- Event descriptions and images submitted for annotation are sent to Anthropic
+- Server mode bills the operator's Claude Platform on AWS workspace; BYOK mode
+  bills the caller's own Anthropic account
+- Local inference is not supported; every annotation involves an external API call
 
 ## Security Scanning
 

debug_workflow.py

@@ -22,6 +22,7 @@ Welcome to the HED-BOT documentation! This directory contains comprehensive guid
 - **[Docker Updates](deployment/docker-updates.md)** - Updating Docker deployments
 - **[Docker Architecture](deployment/docker-architecture.md)** - Docker architecture details
 - **[Claude Platform on AWS](deployment/claude-platform-aws.md)** - Configuring the Anthropic Claude API via AWS Marketplace
+- **[Prompt Caching and Usage Reporting](prompt-caching.md)** - What caches, what it saves, and how to verify it
 
 ### 🛠️ Development
 - **[Contributing](development/contributing.md)** - How to contribute, issue labeling system