Skip to content

feat(llm): add OrcaRouter provider 🤖🤖🤖🤖 - #2756

Open
XiaoHuo888-hue wants to merge 1 commit into
Giskard-AI:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider
Open

feat(llm): add OrcaRouter provider 🤖🤖🤖🤖#2756
XiaoHuo888-hue wants to merge 1 commit into
Giskard-AI:mainfrom
XiaoHuo888-hue:feat/orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Description

This adds a dedicated OrcaRouter provider to the giskard-llm routing layer, mirroring how the existing named providers (azure/, azure_ai/) are wired. A model string like orcarouter/anthropic/claude-haiku-4.5 now routes through the new orcarouter registry entry to https://api.orcarouter.ai/v1 using an ORCAROUTER_API_KEY.

Why a named provider rather than the generic OpenAI-compatible endpoint? Named providers are the established pattern in this library (azure/, azure_ai/, google/, anthropic/), and named routers such as orcarouter/auto pick an upstream per request, so the registry entry keeps model routing and error attribution consistent with the other named providers. OrcaRouter is an OpenAI-compatible gateway that exposes 150+ models from OpenAI, Anthropic, Google, DeepSeek, Qwen, MiniMax and xAI behind one API key, and it also provides gateway-level security controls for AI agents.

Changes:

  • New libs/giskard-llm/src/giskard/llm/providers/orcarouter.pyOrcaRouterProvider, an OpenAIProvider subclass that defaults base_url to https://api.orcarouter.ai/v1 and reads ORCAROUTER_API_KEY (mirrors the AzureOpenAIProvider pattern).
  • libs/giskard-llm/src/giskard/llm/routing.py — register orcarouter in _PROVIDER_REGISTRY.
  • libs/giskard-llm/README.md — add the orcarouter/ row to the provider reference table.
  • Tests in test_providers.py (transport config, env-key + endpoint defaults) and test_routing.py (model-string parsing, unconfigured registry lookup).

Verification:

  • uv run --directory libs/giskard-llm pytest tests -m "not functional" — 222 passed, 7 skipped.
  • ruff check / ruff format --check — clean; basedpyright --level error — 0 errors.
  • Live: acompletion("orcarouter/openai/gpt-5.5", ...) and aembedding("orcarouter/openai/text-embedding-3-small", ...) against https://api.orcarouter.ai/v1 both returned HTTP 200.

Related Issue

N/A

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Coding agents

This PR was opened by an autonomous agent; per the repo's PR rules the title ends with the required 🤖🤖🤖🤖 marker, and the author read AUTONOMOUS.md before opening it.

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in NumPy format for all the methods and classes that I created or modified.
  • I've updated the uv.lock running uv lock (only applicable when pyproject.toml has been modified)

I'm an engineer on the OrcaRouter team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

1 participant