Support external rerank endpoints

[ErikBPF]

Support external rerank endpoints

Summary

• support Cohere-compatible external /rerank endpoints
• retain the local Xenova reranker as fallback
• allow authenticated and unauthenticated local endpoints
• document base URL, model, key, and timeout configuration
• keep external request failures fail-open

Why

Deployments with centralized multilingual rerankers currently need to patch the
worker because reranking is fixed to the bundled English-oriented Xenova model.
An optional HTTP endpoint allows BGE-class or provider-hosted rerankers without
changing default behavior.

Fixes #1081.

Configuration

RERANK_ENABLED=true
RERANK_BASE_URL=http://localhost:4000
RERANK_API_KEY=                    # optional; falls back to OPENAI_API_KEY
RERANK_MODEL=bge-reranker-v2-m3
RERANK_TIMEOUT_MS=30000

When RERANK_BASE_URL is absent, behavior remains unchanged and the local
reranker is used.

Validation

• npx vitest run test/reranker.test.ts: 11 passed
• npm run build: passed
• npm run skills:check: 15 skills passed
• npm test under Node 20: 1,422 passed
• npm test under Node 22: 1,422 passed
• npm run test:integration: 17 passed
• git diff --check main...HEAD: passed

Review notes

• one signed-off commit
• no generated build artifacts committed
• no changelog changes

Summary by CodeRabbit

• New Features
  • Added optional external reranking for hybrid search with configurable provider endpoint, model, and timeout.
  • Availability now accounts for an externally configured reranker (not just local pipeline readiness).
  • Automatically prioritizes external reranking when available; otherwise falls back to local reranking or original results.
• Documentation
  • Updated environment-variable docs with commented reranker settings (enabled flag, endpoint, API key/model, timeout).
• Tests
  • Expanded coverage for external reranking success, auth handling, malformed/timeout/network error fail-open behavior, and invalid rerank outputs.

[vercel]

@ErikBPF is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1daf48b7-977f-4655-8a8e-12c3890a0e8f

📥 Commits

Reviewing files that changed from the base of the PR and between 80eb4b4 and 5f3fb65.

📒 Files selected for processing (2)

• src/state/reranker.ts
• test/reranker.test.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/state/reranker.ts

---

📝 Walkthrough

Walkthrough

The reranker now supports configurable external /rerank providers before the local Transformers pipeline. Responses are validated, scores reorder candidates, and failures preserve the original results. Configuration documentation and comprehensive tests were added.

Changes

External reranking

Layer / File(s)	Summary
External rerank configuration .env.example, README.md	Adds commented settings for reranking enablement, endpoint, API key, model, and timeout.
External rerank request and fallback src/state/reranker.ts	Calls the configured endpoint, validates returned indices and scores, applies rerank ordering, and preserves the local path or original results when external processing fails.
External rerank validation coverage test/reranker.test.ts	Tests authenticated and unauthenticated requests, ordering, timeout defaults, malformed or unsuccessful responses, request errors, invalid indices, and test-state cleanup.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant rerank
  participant RerankEndpoint
  participant LocalPipeline
  Client->>rerank: rerank candidate documents
  rerank->>RerankEndpoint: POST /rerank
  RerankEndpoint-->>rerank: ranked indices and scores
  rerank-->>Client: reordered results
  rerank->>LocalPipeline: use local reranking when external result is unavailable
  LocalPipeline-->>rerank: locally scored results

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding support for external rerank endpoints.
Linked Issues check	✅ Passed	The changes implement configurable external /rerank support, preserve local fallback, fail open, and validate provider indices as requested in #1081.
Out of Scope Changes check	✅ Passed	The modified files stay focused on rerank configuration, implementation, docs, and tests, with no obvious unrelated additions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/state/reranker.ts`:
- Around line 13-22: Prevent apiKey in the rerank request from unconditionally
falling back to OPENAI_API_KEY. Update the baseUrl/apiKey selection so the
OpenAI credential is used only when RERANK_BASE_URL matches the configured
OpenAI endpoint; otherwise require RERANK_API_KEY and send no fallback
credential to third-party hosts.
- Around line 34-36: Validate and normalize RERANK_TIMEOUT_MS once before
constructing the AbortSignal in the reranking flow, accepting only a positive
finite numeric timeout and defaulting invalid, zero, or missing values to 30_000
milliseconds. Use the validated value in AbortSignal.timeout instead of passing
the raw environment string.

In `@test/reranker.test.ts`:
- Around line 39-47: Reset all rerank-related environment variables, including
OPENAI_API_KEY, in a beforeEach hook as well as afterEach. Update the test setup
around the existing lifecycle hooks so each test starts with a clean environment
and remains independent of ambient variables or execution order.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 14c7b9dd-4b05-4b04-b0c2-26cad90dca9d

📥 Commits

Reviewing files that changed from the base of the PR and between a8e7d19 and 80eb4b4.

📒 Files selected for processing (4)

• .env.example
• README.md
• src/state/reranker.ts
• test/reranker.test.ts