fix: replace OpenAI SDK with raw HTTP to avoid Cloudflare WAF blocking, broaden exception handling - #880
Conversation
…g, broaden exception handling, align vector dimensions at startup Problem - The OpenAI Python SDK (AsyncOpenAI) injects x-stainless-* telemetry headers into every request. Cloudflare-proxied OpenAI-compatible endpoints block these, causing embedding generation to fail with PermissionDeniedError. These exceptions are not ValueError subclasses, so they bypassed the except ValueError handler and became unhandled HTTP 500 errors. - pgvector column dimension mismatch between migration default (1536) and runtime config caused unhandled DataError on insert. - Missing broad exception catch around embedding calls in document CRUD. Changes - src/embedding_client.py: Replace AsyncOpenAI with raw httpx.AsyncClient POST requests, sending only Authorization and Content-Type headers. - src/crud/document.py: Add DataError alongside IntegrityError; add broad except Exception catch around embedding call → ValidationException (422). - docker/entrypoint.sh: Add scripts/configure_embeddings.py --yes step after migration to auto-align pgvector dimensions to current config. - tests/: Update all hardcoded 1536-dimension test vectors to use settings.EMBEDDING.VECTOR_DIMENSIONS across 6 test files (47 occurrences). Testing - Manual: create conclusion with session_id, without session_id, self-conclusion all return HTTP 201 instead of 500. - Unit: 27/27 conclusion route tests pass. - CI: ruff check, basedpyright, pytest pass.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe startup flow configures embeddings after database provisioning. OpenAI embeddings use direct HTTP requests, document error handling is expanded, and default vector dimensions change from 1536 to 1024. Tests now use configured dimensions and HTTP request fakes. ChangesConfigurable embedding dimensions and OpenAI HTTP client
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant EntryPoint
participant ProvisionDB
participant ConfigureEmbeddings
EntryPoint->>ProvisionDB: start provisioning
ProvisionDB->>ProvisionDB: initialize database
ProvisionDB->>ConfigureEmbeddings: execute --yes
ConfigureEmbeddings-->>ProvisionDB: return output and exit code
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
src/crud/document.py (1)
896-899: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winBroad
except Exceptionconflates client errors with transient/infra failures.Wrapping any embedding-call exception (timeouts, connection resets, provider 5xx/rate-limit errors) as a
ValidationException(422) tells the client their input was invalid, when the real cause may be a transient dependency failure the client should retry rather than "fix." This trades one wrong status code (500) for another kind of misleading one (422).Consider catching narrower transport/HTTP exception types (e.g., from
httpx) and mapping them to a more appropriate status (e.g., 503) while reservingValidationExceptionfor genuinely malformed input.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/crud/document.py` around lines 896 - 899, The broad exception handling around the embedding generation block in document CRUD is mapping all failures to ValidationException, which incorrectly treats transport/provider outages as bad input. Narrow the except path around the embedding call to catch only genuine validation/malformed-input cases for ValidationException, and handle HTTP/transport failures separately in the relevant embedding helper or method with a more appropriate retryable/server-side exception (for example, using the document embedding flow in src/crud/document.py and the embedding-call wrapper that raises the current error).src/embedding_client.py (1)
263-282: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate request-building/error-handling logic between single and batch embed paths.
embed()and the OpenAI branch of_process_batch()both construct the same headers dict, POST to the same endpoint, checkresp.is_success, and parseresp.json()["data"]almost identically. Consider extracting a small private helper (e.g._openai_post_embeddings(http_client, payload)) to avoid maintaining two copies of this logic (e.g., if auth header format or error message ever needs to change).♻️ Proposed extraction
+ async def _openai_post_embeddings( + self, http_client: httpx.AsyncClient, payload: dict[str, Any] + ) -> dict[str, Any]: + resp = await http_client.post( + f"{self._openai_base_url}/embeddings", + json=payload, + headers={ + "Authorization": f"Bearer {self._openai_api_key}", + "Content-Type": "application/json", + }, + ) + if not resp.is_success: + raise RuntimeError(f"Embedding API error: {resp.status_code} {resp.text}") + return resp.json()Then call
data = await self._openai_post_embeddings(http_client, payload)from bothembed()and_process_batch().Also applies to: 478-500
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/embedding_client.py` around lines 263 - 282, The OpenAI embedding request logic is duplicated in both embed() and the OpenAI branch of _process_batch(), including headers, POST request, success check, and JSON parsing. Extract this into a small private helper such as _openai_post_embeddings(http_client, payload) inside EmbeddingClient, then have both embed() and _process_batch() call it so the auth header, endpoint, and error handling stay in one place.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docker/entrypoint.sh`:
- Around line 7-10: The startup script currently runs the DDL-capable
configure_embeddings.py --yes step unconditionally in entrypoint execution,
which can race across multiple replicas. Move this schema-altering embedding
configuration out of the container entrypoint and into a one-off migration/init
job or equivalent singleton startup path, using the configure_embeddings.py flow
only where it will run once per deployment rather than per app replica.
In `@src/crud/document.py`:
- Around line 1046-1049: The shared except block in the document creation flow
is too broad because it also catches DataError/IntegrityError from later
sync-state commits, causing a misleading “Failed to create observations” message
after the documents have already been committed. Narrow the handler around the
initial insert in the document creation path, or update the ValidationException
message in the create_observations logic to explicitly cover both document
creation and sync-state updates so the error text matches where the failure can
occur.
In `@src/embedding_client.py`:
- Line 9: The OpenAI test monkeypatches are targeting
src.embedding_client.AsyncOpenAI, but that symbol no longer exists and causes
AttributeError during test setup. Update the helpers in
tests/llm/test_embedding_client.py, especially _build_openai_client and related
setup, to patch httpx.AsyncClient behavior or use an injected fake client
instead of monkeypatching AsyncOpenAI. Keep the test coverage focused on the
embedding client’s request/response handling through the actual
httpx.AsyncClient path.
---
Nitpick comments:
In `@src/crud/document.py`:
- Around line 896-899: The broad exception handling around the embedding
generation block in document CRUD is mapping all failures to
ValidationException, which incorrectly treats transport/provider outages as bad
input. Narrow the except path around the embedding call to catch only genuine
validation/malformed-input cases for ValidationException, and handle
HTTP/transport failures separately in the relevant embedding helper or method
with a more appropriate retryable/server-side exception (for example, using the
document embedding flow in src/crud/document.py and the embedding-call wrapper
that raises the current error).
In `@src/embedding_client.py`:
- Around line 263-282: The OpenAI embedding request logic is duplicated in both
embed() and the OpenAI branch of _process_batch(), including headers, POST
request, success check, and JSON parsing. Extract this into a small private
helper such as _openai_post_embeddings(http_client, payload) inside
EmbeddingClient, then have both embed() and _process_batch() call it so the auth
header, endpoint, and error handling stay in one place.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e025854b-a406-47db-8d18-eafeb2634433
📒 Files selected for processing (9)
docker/entrypoint.shsrc/crud/document.pysrc/embedding_client.pytests/alembic/revisions/test_b8183c5ffb48_codify_document_level_and_times_derived.pytests/crud/test_document.pytests/deriver/test_vector_reconciliation.pytests/llm/test_embedding_client.pytests/routes/test_conclusions.pytests/utils/test_agent_tools.py
…ty, helper extraction, test fix Changes per review: 1. docker/entrypoint.sh — Move DDL configure_embeddings.py --yes into scripts/provision_db.py (DB init runs once) via subprocess.run non-fatal, avoiding race condition on ALTER COLUMN across replicas. 2. scripts/provision_db.py — Add non-fatal subprocess.run call to configure_embeddings.py --yes after alembic migration completes. 3. src/crud/document.py — Split try/except: document insert wrapped with (DataError, IntegrityError) → ValidationException; sync-state updates moved outside the insert try/except block. Broaden embedding-error handling: ValueError → 422, httpx.HTTPStatusError → 503, httpx.ConnectError/TimeoutException/RemoteProtocolError → 503, generic Exception → 500. 4. src/embedding_client.py — Extract _openai_post_embeddings(http_client, payload) helper to eliminate duplicate POST logic between embed() and _process_batch(). 5. tests/llm/test_embedding_client.py — Replace monkeypatched AsyncOpenAI with monkeypatched httpx.AsyncClient.post using httpx.Response fixtures, fixing AttributeError from removed class.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tests/llm/test_embedding_client.py (1)
315-326: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate
fake_postmocking logic across tests.Three tests (
test_simple_batch_embed_respects_token_budget_per_request,test_simple_batch_embed_rejects_oversized_input,test_prepare_chunks_returns_ordered_chunks) reimplement inlinefake_post/client-construction logic that closely mirrors_build_openai_client. Consider extending_build_openai_client(e.g. optionalmax_input_tokens/max_tokens_per_requestparams) to cover these cases and reduce duplicated mock setup.Also applies to: 353-356, 379-382
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/llm/test_embedding_client.py` around lines 315 - 326, The three embedding-client tests are duplicating the same `fake_post` and client setup logic instead of reusing the shared helper. Update `_build_openai_client` in the embedding client test utilities to accept the needed optional parameters such as `max_input_tokens` and/or `max_tokens_per_request`, then switch `test_simple_batch_embed_respects_token_budget_per_request`, `test_simple_batch_embed_rejects_oversized_input`, and `test_prepare_chunks_returns_ordered_chunks` to use that helper so the mocked `httpx.AsyncClient.post` setup is centralized.src/crud/document.py (1)
859-862: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocstring
Raisessection is now incomplete.The new embedding-failure paths (Lines 898-912) raise
HonchoExceptionfor provider HTTP errors and transient connectivity issues, but the docstring only documentsResourceNotFoundExceptionandValidationException.📝 Proposed docstring update
Raises: ResourceNotFoundException: If any session or peer is not found - ValidationException: If embedding generation fails or integrity constraint is violated + ValidationException: If embedding validation fails or a data/integrity constraint is violated + HonchoException: If the embedding provider returns an error or is unreachableAs per coding guidelines, "In Python code, use Google-style docstrings."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/crud/document.py` around lines 859 - 862, Update the Google-style docstring for the method in document.py that handles embedding generation so the Raises section includes HonchoException alongside ResourceNotFoundException and ValidationException. Add a brief note that HonchoException is raised for provider HTTP errors and transient connectivity failures in the embedding-generation paths, and keep the documentation aligned with the new error behavior in the embedding workflow.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/crud/document.py`:
- Around line 977-988: The pgvector-only sync-state update in the document flow
is missing the same error handling used in the external-vector-store branch, so
a transient failure in db.execute or db.commit can still bubble up as a 500.
Wrap the no-external-vector-store path in the same kind of try/except handling
used around upsert_many/db.execute/commit for the external branch in
src/crud/document.py, and make sure the exception path logs the failure and
returns a controlled response instead of propagating it.
---
Nitpick comments:
In `@src/crud/document.py`:
- Around line 859-862: Update the Google-style docstring for the method in
document.py that handles embedding generation so the Raises section includes
HonchoException alongside ResourceNotFoundException and ValidationException. Add
a brief note that HonchoException is raised for provider HTTP errors and
transient connectivity failures in the embedding-generation paths, and keep the
documentation aligned with the new error behavior in the embedding workflow.
In `@tests/llm/test_embedding_client.py`:
- Around line 315-326: The three embedding-client tests are duplicating the same
`fake_post` and client setup logic instead of reusing the shared helper. Update
`_build_openai_client` in the embedding client test utilities to accept the
needed optional parameters such as `max_input_tokens` and/or
`max_tokens_per_request`, then switch
`test_simple_batch_embed_respects_token_budget_per_request`,
`test_simple_batch_embed_rejects_oversized_input`, and
`test_prepare_chunks_returns_ordered_chunks` to use that helper so the mocked
`httpx.AsyncClient.post` setup is centralized.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 8571742b-27d4-427d-9119-840a5da7b79c
📒 Files selected for processing (5)
docker/entrypoint.shscripts/provision_db.pysrc/crud/document.pysrc/embedding_client.pytests/llm/test_embedding_client.py
✅ Files skipped from review due to trivial changes (1)
- docker/entrypoint.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- src/embedding_client.py
…cstring, wrap pgvector sync in try/except Changes: - tests/llm/test_embedding_client.py: Extend _build_openai_client helper with optional max_input_tokens/max_tokens_per_request params and refactor 3 tests (test_simple_batch_embed_respects_token_budget_per_request, test_simple_batch_embed_rejects_oversized_input, test_prepare_chunks_returns_ordered_chunks) to use it, eliminating duplicate fake_post mocking logic. - src/crud/document.py: Update Google-style docstring Raises section to include HonchoException for provider HTTP errors and transient connectivity failures. Wrap the pgvector-only sync-state update path (no external vector store) in try/except with logger.exception to prevent transient DB failures from bubbling up as HTTP 500.
…ate error handler The pgvector-only path in create_observations was missing db.rollback() in the except block, leaving the session in a potentially broken state after a failed commit. Also adds workspace_name to the log message for better observability — matching the pattern used in the external vector store path.
- Update VECTOR_DIMENSIONS and DIMENSIONS defaults in src/config.py: 1536 → 1024 - Update config templates: .env.template, config.toml.example - Fix docstring in src/dreamer/surprisal.py - Fix hardcoded 1536 vector dimensions in tests to use settings.EMBEDDING.VECTOR_DIMENSIONS - Update test fixtures and restore dimensions in test_configure_embeddings, test_embedding_validator, test_models_vector_dim - Update integration test fake embedding fixture to 1024 dimensions All 52 tests pass: test_document (21), test_representation_manager (13), test_embedding_validator (10), test_models_vector_dim (2), test_configure_embeddings (6)
6c8c27b to
3b6bbe2
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/config.py`:
- Line 751: Restore EMBEDDING.VECTOR_DIMENSIONS to the existing 1536 default so
it remains compatible with the pgvector migrations and populated deployments. Do
not introduce a dimension migration unless it is fully implemented and tested;
preserve the positive-value validation.
- Line 751: The default VECTOR_DIMENSIONS path must send dimensions=1024 to the
embedding API instead of omitting the parameter in auto mode. Update
resolve_send_dimensions() or its caller so the Pydantic default is treated as an
explicit request, and add coverage verifying a default configuration sends the
1024 dimension.
In `@src/dreamer/surprisal.py`:
- Line 362: Update the return-shape documentation for _extract_embeddings to
avoid hardcoding 1024; describe embeddings as shape (N, D) or reference
EmbeddingSettings.VECTOR_DIMENSIONS, while preserving the configurable dimension
contract.
In `@tests/test_models_vector_dim.py`:
- Around line 51-53: Update test_models_uses_default_1024_when_no_env_override
and its _run_in_fresh_interpreter invocation to remove
EMBEDDING_VECTOR_DIMENSIONS from the child environment before asserting the
default dimensions, while preserving the existing subprocess-based test flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 9ff0e5ae-3e1b-4bd4-b8a5-5cfda619f9bd
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (11)
.env.templateconfig.toml.examplesrc/config.pysrc/crud/document.pysrc/dreamer/surprisal.pytests/crud/test_document.pytests/integration/test_representation.pytests/scripts/test_configure_embeddings.pytests/startup/test_embedding_validator.pytests/test_models_vector_dim.pytests/utils/test_agent_tools.py
🚧 Files skipped from review as they are similar to previous changes (3)
- tests/utils/test_agent_tools.py
- tests/crud/test_document.py
- src/crud/document.py
| default_factory=_MODEL_CONFIG_DEFAULT | ||
| ) | ||
| VECTOR_DIMENSIONS: Annotated[int, Field(default=1536, gt=0)] = 1536 | ||
| VECTOR_DIMENSIONS: Annotated[int, Field(default=1024, gt=0)] = 1024 |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | rg '(^|/)src/config\.py$|test|migration|migrat' | head -200
echo "== config outline =="
ast-grep outline src/config.py --view compact | sed -n '1,220p' || true
echo "== VectorDimensions context =="
nl -ba src/config.py | sed -n '720,770p'
echo "== search VECTOR_DIMENSIONS and dimension provisioning tests =="
rg -n "VECTOR_DIMENSIONS|vector dimensions|embedding dimensions|vector.*dim|dim.*vector|CREATE .*VECTOR|ALTER .*VECTOR|1536|1024|provisioning|provision" -S --glob '*.py' . | sed -n '1,260p'Repository: plastic-labs/honcho
Length of output: 9165
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== src/config.py VECTOR_DIMENSIONS context =="
sed -n '720,770p' src/config.py | cat -n
echo "== VECTOR_DIMENSIONS and relevant tests/search =="
rg -n "VECTOR_DIMENSIONS|vector dimensions|embedding dimensions|vector.*dim|dim.*vector|CREATE .*VECTOR|ALTER .*VECTOR|1536|1024|provisioning|provision" -S --glob '*.py' . | cat
echo "== relevant database vector references =="
rg -n "VECTOR|pgvector|embeddings|MessageEmbedding|messageembeddings|dimension" -S --glob '*.py' . | sed -n '1,260p'Repository: plastic-labs/honcho
Length of output: 50375
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== configure_embeddings.py key sections =="
sed -n '120,225p' scripts/configure_embeddings.py | cat -n
printf '\n-- script header and target_dim sections --\n'
sed -n '1,35p' scripts/configure_embeddings.py | cat -n
sed -n '380,470p' scripts/configure_embeddings.py | cat -n
echo "== startup embedding validator =="
sed -n '60,180p' src/startup/embedding_validator.py | cat -n
echo "== main startup validator usage =="
sed -n '100,120p' src/main.py | cat -n
echo "== test vector dimension probe excerpts =="
sed -n '120,175p' tests/startup/test_embedding_validator.py | cat -n
sed -n '130,210p' tests/scripts/test_configure_embeddings.py | cat -n
echo "== migration Vector(1536) exact lines =="
rg -n "Vector\(1536\)" migrations tests/scripts/test_configure_embeddings.py src/models.py -S | catRepository: plastic-labs/honcho
Length of output: 22592
Keep the default embedding dimension at 1536 or ship a tested migration.
src/config.py now defaults EMBEDDING.VECTOR_DIMENSIONS to 1024 while the pgvector migrations still create Vector(1536). Existing 1536 deployments with populated documents/message_embeddings rows cannot update via scripts/configure_embeddings.py because it refuses dimension changes on non-null embeddings, and the API startup validator will reject the mismatch.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/config.py` at line 751, Restore EMBEDDING.VECTOR_DIMENSIONS to the
existing 1536 default so it remains compatible with the pgvector migrations and
populated deployments. Do not introduce a dimension migration unless it is fully
implemented and tested; preserve the positive-value validation.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Locate src/config.py and relevant tests =="
git ls-files | rg '(^|/)src/config\.py$|test_configure_embeddings|embedding|configure' | head -200
echo
echo "== config.py relevant sections =="
if [ -f src/config.py ]; then
wc -l src/config.py
sed -n '720,770p' src/config.py
fi
echo
echo "== Search resolve_send_dimensions and VECTOR_DIMENSIONS usages =="
rg -n "resolve_send_dimensions|VECTOR_DIMENSIONS|dimensions" src tests 2>/dev/null | head -300Repository: plastic-labs/honcho
Length of output: 25540
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== src/config.py EmbeddingSettings section =="
sed -n '700,800p' src/config.py
echo
echo "== src/embedding_client.py OpenAI request dimension handling =="
sed -n '130,295p' src/embedding_client.py
sed -n '435,500p' src/embedding_client.py
echo
echo "== tests for default dimension resolver/client payload =="
sed -n '240,260p' tests/llm/test_embedding_client.py
sed -n '148,198p' tests/llm/test_embedding_client.py
sed -n '1,120p' tests/scripts/test_configure_embeddings.pyRepository: plastic-labs/honcho
Length of output: 20371
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== OpenAI embedding model dimension defaults in config/tests =="
rg -n "text-embedding-3-small|text-embedding-3-large|OPENAI|dimensions|vector_dimensions" src config.py tests/llm test_model_config tests scripts/migrations 2>/dev/null | head -240 || true
echo
echo "== Behavioral probe: Pydantic default field presence in model_fields_set =="
python3 - <<'PY'
from pathlib import Path
try:
from pydantic import BaseModel, Field, model_validator
except Exception as e:
print(f"pydantic import unavailable: {e}")
raise SystemExit
class Base(BaseModel):
pass
class Config(Base):
MODE: str = "auto"
class Settings(Base):
model_config = {"env_nested_delimiter": "__"}
MODEL_CONFIG: Config = Field(default_factory=lambda: Config())
VECTOR_DIMENSIONS: int = Field(default=1024, gt=0) = 1024
`@model_validator`(mode="before")
`@classmethod`
def merge_defaults(cls, data):
return data
default = Settings()
explicit = Settings(VECTOR_DIMENSIONS=768)
env_like = Settings(__dict__={"VECTOR_DIMENSIONS": 768})
print("default resolve false branch:", default.VECTOR_DIMENSIONS, "in model_fields_set:", "VECTOR_DIMENSIONS" in default.model_fields_set)
print("explicit resolve true branch:", explicit.VECTOR_DIMENSIONS, "in model_fields_set:", "VECTOR_DIMENSIONS" in explicit.model_fields_set)
print("env-like resolve true branch:", env_like.VECTOR_DIMENSIONS, "in model_fields_set:", "VECTOR_DIMENSIONS" in env_like.model_fields_set)
PY
echo
echo "== Check if target model has built-in default documented in tests/config comments =="
rg -n "text-embedding-3-small|text-embedding-3-large|1536|1536 dimensions|dimensions=1024|dimensions=1536" . --glob '!tests/unified/**' 2>/dev/null | head -200 || trueRepository: plastic-labs/honcho
Length of output: 25346
🌐 Web query:
OpenAI text-embedding-3-small dimensions default API returns 1024 dimensions parameter
💡 Result:
The default output dimension for the OpenAI text-embedding-3-small model is 1536 [1]. While the model produces 1536 dimensions by default, developers can control the size of the embedding vector by using the optional dimensions parameter in the API request [1][2]. This parameter allows users to shorten the embedding to a lower dimension, which can be useful for fitting into vector databases with specific size constraints or for optimizing performance and cost [1][3]. The dimensions parameter is supported specifically in the text-embedding-3 and later models [2][4].
Citations:
- 1: https://developers.openai.com/api/docs/guides/embeddings
- 2: https://developers.openai.com/api/reference/resources/embeddings/methods/create
- 3: https://openai.com/index/new-embedding-models-and-api-updates/
- 4: https://docs.github.com/en/rest/models/embeddings
🌐 Web query:
OpenAI text-embedding-3-small dimensions parameter default 1024
💡 Result:
The default number of dimensions for the OpenAI text-embedding-3-small model is 1536 [1][2][3]. While the model defaults to 1536 dimensions, it supports a dimensions parameter that allows you to reduce the output vector length without needing to retrain the model [1][4][3]. You can specify a lower number of dimensions (for example, 1024, 768, 512, or 256) by passing the dimensions parameter in your API request [1][2][3][5]. This feature is useful for optimizing storage, memory usage, and search latency in vector databases, especially when those systems have constraints on the maximum supported embedding length [1][4][2].
Citations:
- 1: https://developers.openai.com/api/docs/guides/embeddings
- 2: https://crazyrouter.com/en/blog/text-embedding-3-small-dimensions-explained-20260605044315
- 3: https://tokenmix.ai/blog/text-embedding-3-small-developer-guide-2026
- 4: https://openai.com/index/new-embedding-models-and-api-updates/
- 5: https://crazyrouter.com/en/blog/text-embedding-3-small-api-guide-20260223091247
Ensure the default dimension is sent to the embedding API.
resolve_send_dimensions() returns False in auto mode when VECTOR_DIMENSIONS is only the Pydantic default, while OpenAI’s text-embedding-3-small native/legacy shape is 1536. A default request can therefore provision 1024-dimensional columns but omit dimensions, causing OpenAI-compatible embedding clients to return 1536-dimensional vectors or reject the local dimension check. Make the default path send dimensions=1024, or adjust resolve_send_dimensions() to send defaults as well, and add default-dimension request coverage.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/config.py` at line 751, The default VECTOR_DIMENSIONS path must send
dimensions=1024 to the embedding API instead of omitting the parameter in auto
mode. Update resolve_send_dimensions() or its caller so the Pydantic default is
treated as an explicit request, and add coverage verifying a default
configuration sends the 1024 dimension.
|
|
||
| Returns: | ||
| np.ndarray of shape (N, 1536) containing embeddings | ||
| np.ndarray of shape (N, 1024) containing embeddings |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Keep the embedding shape contract dimension-agnostic.
EmbeddingSettings.VECTOR_DIMENSIONS is configurable, so _extract_embeddings can return (N, 768) or another valid shape. Document the result as (N, D) or reference the configured dimension instead of hardcoding 1024.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/dreamer/surprisal.py` at line 362, Update the return-shape documentation
for _extract_embeddings to avoid hardcoding 1024; describe embeddings as shape
(N, D) or reference EmbeddingSettings.VECTOR_DIMENSIONS, while preserving the
configurable dimension contract.
| def test_models_uses_default_1024_when_no_env_override() -> None: | ||
| dims = _run_in_fresh_interpreter({}) | ||
| assert dims == {"message_embedding_dim": 1536, "document_dim": 1536} | ||
| assert dims == {"message_embedding_dim": 1024, "document_dim": 1024} |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Isolate the “no env override” test from ambient settings.
The subprocess environment inherits os.environ, so an externally set EMBEDDING_VECTOR_DIMENSIONS can override the default and make this assertion misleading. Remove that variable from the child environment before testing the default.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_models_vector_dim.py` around lines 51 - 53, Update
test_models_uses_default_1024_when_no_env_override and its
_run_in_fresh_interpreter invocation to remove EMBEDDING_VECTOR_DIMENSIONS from
the child environment before asserting the default dimensions, while preserving
the existing subprocess-based test flow.
1 participant
Summary
Fix
POST /v3/workspaces/{workspace_id}/conclusionsreturning HTTP 500 due to embedding API failures and unhandled database exceptions.Root Cause
The OpenAI Python SDK (
AsyncOpenAI) automatically injectsx-stainless-*telemetry headers into every request. When using a Cloudflare-proxied OpenAI-compatible endpoint (e.g.,api.hungcuong.dev), Cloudflare's WAF blocks these non-standard headers, causing embedding generation to fail withPermissionDeniedError. Since the SDK's exception types (APIConnectionError,APIStatusError, etc.) are not subclasses ofValueError, they bypassed the existingexcept ValueErrorhandler and propagated to the global exception handler as HTTP 500.Secondary issue: a pgvector column dimension mismatch between the deployment config and migration default also caused an unhandled
DataErroron insert.Changes
src/embedding_client.py— Replace OpenAI SDK with raw HTTPProblem:
AsyncOpenAI(api_key=..., base_url=...)sendsx-stainless-arch,x-stainless-lang,x-stainless-package-version,x-stainless-runtime,x-stainless-runtime-versionheaders. Cloudflare WAF at common OpenAI-compatible reverse proxies rejects these.Fix: Replace
AsyncOpenAIwithhttpx.AsyncClientand make raw POST requests to{base_url}/embeddingswith onlyAuthorizationandContent-Typeheaders. This applies to bothembed()(single query) and_process_batch()(batch) code paths.src/crud/document.py— Broaden exception handlingDataErroralongsideIntegrityErrorin the DB commit catch block — covers pgvector dimension mismatches.except Exceptionaround the embedding call — any embedding failure (network, API, auth) now becomes aValidationException(422) instead of 500.docker/entrypoint.sh— Auto-align pgvector dimensionsMigration
b8183c5ffb48hardcodesVector(1536). If the deployment config uses a different dimension, inserts fail withDataError. Addedscripts/configure_embeddings.py --yesstep at startup to ALTER the vector column to match the current config.Test fixes
Replaced all hardcoded
[0.x] * 1536test vectors with[0.x] * settings.EMBEDDING.VECTOR_DIMENSIONSacross 6 test files (47 occurrences) to match the runtime configuration.Testing
session_id✅session_id✅observer_id == observed_id) ✅Checklist
ruff check,basedpyright)pytest tests/routes/test_conclusions.py)docker compose build)Summary by CodeRabbit
New Features
Bug Fixes
Tests
httpx.