Phase 2: Anthropic transport + universal prompt caching

[neuromechanist]

Part of epic #155.

Goal: Settle the Anthropic transport and make prompt caching apply to every agent's static system prompt, not just annotation.

Scope

• Transport research: does LiteLLM expose a native provider/router for AWS-external Anthropic (non-Bedrock, the direct-AWS-billing endpoint)? Record findings on this issue. Adopt it if it exists; otherwise keep explicit api_base + workspace header (or the official Anthropic SDK via langchain-anthropic).
• Universal caching: apply cache_control to the static system prefix of the vision, evaluation/judge, assessment, feedback, and keyword agents (today only annotation caches via CachingLLMWrapper, src/utils/openrouter_llm.py:100).
• Audit each agent's system prompt is static per schema/config (no per-request content leaking into the cached prefix); move any dynamic content into the user message.
• Fix/confirm is_cacheable_model() for native claude- ids.

Definition of Done

• Cache write + read observable (usage metadata) for all agent roles on the Anthropic path.
• A written finding on LiteLLM AWS-Anthropic (non-Bedrock) support, with the adopted transport decision recorded here.

[neuromechanist]

Transport finding (Phase 2 deliverable)

Adopted: langchain-anthropic (ChatAnthropic) with base_url pointed at the
AWS-external endpoint and the workspace id sent via default_headers
(anthropic-workspace-id). The official Anthropic SDK sits underneath it.

LiteLLM: not used, and not needed. The question in this issue was whether LiteLLM
exposes a native provider/router for the AWS-external (non-Bedrock) endpoint. It does
not need to: the endpoint is the ordinary Anthropic Messages API at a different base URL
plus one header, so the official SDK reaches it with two constructor arguments. Adding
LiteLLM back would insert a translation layer between HEDit and the API whose caching
and thinking parameters we depend on, for no capability we lack. The OpenRouter/LiteLLM
modules were removed in #162.

BYOK keys are pinned to https://api.anthropic.com explicitly, because ChatAnthropic
would otherwise inherit the server's ANTHROPIC_BASE_URL from the process environment
and send a first-party key to the AWS endpoint, which rejects it.

Universal caching: measured, and it does not apply the way this issue assumed

Measured against the AWS endpoint with the Messages API count_tokens endpoint
(2026-08-20, claude-haiku-4-5):

Role	System prompt	Caches on Haiku 4.5?
Annotation	21,811 tokens	Yes
Evaluation	623 tokens	No
Assessment	266 tokens	No
Feedback summarizer	241 tokens	No
Keyword extraction	186 tokens	No
Vision (task prompt, sent as a user message)	51 tokens	No

Haiku 4.5 will not create a cache entry below 4096 tokens (Sonnet 5: 1024). Five of
the six prompts are 7-80x below that. A short prefix keeps its cache_control marker,
creates nothing, is charged no write premium, and reports cache_creation: 0 with no
error, which is why this needed measuring rather than assuming.

So the scope item "apply cache_control to the static prefix of the vision, evaluation,
assessment, feedback, and keyword agents" cannot produce cache hits by adding markers.
Padding those prompts to 4096 tokens would cost more per call than caching saves. The
markers stay in place (free no-ops that would start working on a large prefix, or on a
model with a lower minimum), and the honest conclusion is recorded in
docs/prompt-caching.md rather than implied by their presence.

The audit item did hold up: every role's system prompt is static per schema/config, and
no per-request content leaks into a cached prefix. no_extend and the schema version
legitimately fork the annotation prefix into separate cache lanes.
tests/test_annotation_agent.py::TestCachedPrefixStability now asserts byte-identity so
a future timestamp or request id in the prefix fails a test instead of silently costing
90% more.

is_cacheable_model() no longer exists; #162 replaced it with an allowlist of Anthropic
models, and caching is on by default for every role.

Cache activity is now observable

The Definition of Done asked for cache write and read to be observable from usage
metadata. src/utils/llm_usage.py records per-role tokens, cache reads/writes, and cost
from every response, surfaced in four places: the CLI report, the usage field on API
responses (and the stream result event), GET /metrics for server-wide totals by role
and model, and telemetry events (whose token/cost fields were declared but never
populated).

Observed on two consecutive real requests:

	Request 1 (write)	Request 2 (read)
Input tokens	22,741	22,740
From cache	0	21,805
Cost	$0.029917	$0.004670
Cost without caching	$0.024466	$0.024295
Saved	none	$0.019625 (81%)

The first request in a window costs more than no caching, because of the 1.25x write
premium; break-even is the second request inside 5 minutes. Interactive use spaced
further apart therefore pays the premium repeatedly, so HEDIT_PROMPT_CACHE_TTL now
accepts 1h (2x write, pays off from the third request in the hour). Default stays
5m.

Live verification: tests/test_integration_anthropic.py::TestPromptCachingIntegration
checks both halves (a 21.8k prefix writes then reads back with a saving; a short prefix
produces no cache activity).

PR: #164. Closing this issue when that merges.

[neuromechanist]

Merged into develop via #164 (merge a4f9bfe).

Both Definition-of-Done items are met: the transport decision is recorded above
(langchain-anthropic with base_url plus the workspace header; LiteLLM not used), and
cache write/read is observable per role from usage metadata through the CLI report, the
API usage field, GET /metrics, and telemetry.

The universal-caching scope item is closed with a measurement rather than an
implementation: five of the six agent prompts are below Haiku 4.5's 4096-token minimum
cacheable prefix, so markers cannot make them cache. Documented in
docs/prompt-caching.md alongside the two-request cost comparison and the
HEDIT_PROMPT_CACHE_TTL knob.