Migrate LLM stack to Anthropic Claude via Claude Platform on AWS - #162
Merged
Conversation
Server mode reads ANTHROPIC_API_KEY/ANTHROPIC_BASE_URL/ANTHROPIC_WORKSPACE_ID (workspace header required by the AWS endpoint); BYOK keys go to the first-party API. Offered models: claude-haiku-4-5 (default) and claude-sonnet-5. Replaces langchain-openai/litellm deps with langchain-anthropic. Verified live against both models.
- create_anthropic_workflow/create_byok_workflow/create_vision_agent replace the OpenRouter factories; provider routing and user-id cache lanes removed - BYOK accepts Anthropic keys (sk-ant-) via X-Anthropic-Key; legacy X-OpenRouter-Key header still accepted as transport - Unknown models rejected with 400; server mode no longer needs a per-request key (credentials come from the environment) - Ollama path removed; evaluation judge defaults to Claude Haiku 4.5 - Feedback triage and telemetry defaults updated
- Defaults: claude-haiku-4-5 for annotation, evaluation, and vision; claude-sonnet-5 offered via --model - Provider options removed (single provider now); credentials store anthropic_api_key with HEDIT_ANTHROPIC_API_KEY env override - Standalone mode can run on ANTHROPIC_API_KEY env credentials without a stored BYOK key; client sends X-Anthropic-Key
…thropic-Key Provider plumbing removed from the web UI (single provider now); model values use first-party ids. Worker BYOK detection and header forwarding accept the new X-Anthropic-Key header alongside the legacy one.
- New test_anthropic_llm.py unit tests (normalization, server/BYOK modes, temperature and thinking gating, caching wrapper) - test_integration_openrouter.py renamed to test_integration_anthropic.py, gated on ANTHROPIC_API_KEY - Security, CLI, and endpoint tests updated for X-Anthropic-Key and first-party model ids; removed tests of deleted OpenRouter modules - 440 non-integration tests pass (LSP tests fail locally for an unrelated hed-lsp version issue)
CI now uses the ANTHROPIC_API_KEY secret plus ANTHROPIC_BASE_URL and ANTHROPIC_WORKSPACE_ID repository variables (must be configured on GitHub).
Factory raises RuntimeError when ANTHROPIC_API_KEY is unset in server mode; endpoints map it to 503 while model validation errors stay 400.
- New docs/deployment/claude-platform-aws.md replaces openrouter.md - Ollama/GPU setup removed from deployment guides; Anthropic credential setup and model selection (claude-haiku-4-5 default, claude-sonnet-5 optional) documented throughout - BYOK manual tests rewritten for X-Anthropic-Key with sk-ant keys
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
- 400 mapping for rejected models on all four annotate endpoints; exact 503 with missing server credentials; alias acceptance over HTTP - BYOK key extraction via both X-Anthropic-Key and legacy header - Standalone-mode credential gate (env credentials, no credentials, API mode still requires key) - Replace stale pre-migration telemetry test data; pin the BYOK format-rejection test to 401
Critical: - BYOK LLMs now pin base_url to api.anthropic.com explicitly; ChatAnthropic otherwise inherits the server's ANTHROPIC_BASE_URL from the process env, routing BYOK keys to the AWS endpoint that rejects them (verified live) - Anthropic exceptions map to specific HTTP statuses via a shared classifier across all four annotate endpoints (401 auth, 403 permission, 413 context overflow, 400 bad request, 502 connection, 504/429 kept) Robustness: - Startup validates credentials with a free count_tokens call; failure keeps the server up but marks /health degraded instead of booting a healthy-looking server that 500s on every request - Vision-agent init failure no longer takes down text annotation (503 on image endpoints only) - CachingLLMWrapper raises TypeError on unsupported message types instead of silently relabeling them as user turns - CLI warns once when a legacy OpenRouter key is found in credentials; client gains explicit 400/429 error branches - Frontend streaming handlers surface the backend's error detail instead of a bare HTTP status Cleanup: dead LLM_PROVIDER_PREFERENCE read removed, user_id docstrings no longer claim telemetry recording, obsolete OpenRouter benchmark examples deleted, worker forwards X-OpenRouter-Vision-Model, stale docstrings and comments corrected. All 454 unit tests pass; lifespan integration tests pass against the live endpoint.
Deploying hedit with
|
| Latest commit: |
a12b5df
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://4af71b7e.hedit.pages.dev |
| Branch Preview URL: | https://feature-claude-platform-aws.hedit.pages.dev |
This was referenced Aug 21, 2026
This was referenced Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves all LLM calls from OpenRouter to Anthropic Claude served through the Claude Platform on AWS (Anthropic-operated Messages API, AWS Marketplace billing; not Amazon Bedrock), mirroring the nemar-cli migration.
Model policy
claude-haiku-4-5(default) andclaude-sonnet-5(web UI dropdown option). Opus is not offered.anthropic/claude-haiku-4.5) are accepted as aliases; non-Anthropic models are rejected with HTTP 400.Backend
src/utils/anthropic_llm.pyfactory onlangchain-anthropic(official SDK underneath). Server mode readsANTHROPIC_API_KEY+ANTHROPIC_BASE_URL+ANTHROPIC_WORKSPACE_ID(the endpoint rejects requests without the workspace header). Prompt caching viacache_controlon system messages is preserved.temperature; the factory gates sampling params by model, and mapsdisable_reasoningtothinking: disabledonly where thinking is on by default.sk-ant-) viaX-Anthropic-Key(legacyX-OpenRouter-Keystill accepted as transport) and routes to api.anthropic.com without the workspace header.CLI, frontend, worker, ops
--provideroptions removed; credentials storeanthropic_api_key(HEDIT_ANTHROPIC_API_KEYenv override); standalone mode can run on the server env credentials.X-Anthropic-Key; docker compose, CI workflows, dev scripts, and deployment docs updated (newdocs/deployment/claude-platform-aws.md).Testing
VisionAgent.tests/test_integration_anthropic.pyreplaces the OpenRouter integration tests, gated onANTHROPIC_API_KEY.Required repository configuration (manual)
ANTHROPIC_API_KEYand repository variablesANTHROPIC_BASE_URL,ANTHROPIC_WORKSPACE_IDfor CIOPENROUTER_API_KEY_FOR_TESTINGsecret and repository variable (the variable exposes a key in plaintext) and revoke the OpenRouter keys