feat(stores): add Valkey Search vector store backend by daric93 · Pull Request #10770 · mudler/LocalAI

daric93 · 2026-07-10T20:23:13Z

Description

Adds a new built-in Go gRPC store backend valkey-store that implements the four Stores* RPCs (Set / Get / Delete / Find) against the Valkey Search module (FT.* vector similarity), selectable via the existing per-request backend field on the /stores endpoints. It mirrors the in-memory local-store and adds persistence across restarts plus opt-in HNSW.

No proto change, no HTTP/API change — selected with "backend": "valkey-store" (alias "valkey").
Client: github.com/valkey-io/valkey-go v1.0.76 (official, pure Go, no CGO).
Each vector is a Valkey HASH keyed by hex(little-endian float32); the FT index is created lazily on first Set (FLAT + COSINE by default), and cosine similarity is derived as 1 - distance to match local-store semantics exactly.
Per-namespace, collision-resistant key prefix / index name (sanitize(name) + short sha256).
Config via VALKEY_* env vars (addr, auth, TLS, index algo + HNSW knobs, distance metric, per-command request timeout, mandatory ClientName).

Why

The stores subsystem is designed to be pluggable, but the only shipped backend (local-store) is in-memory and loses all data on restart, and its Find is an O(N) scan. Face/voice biometric registries and the router embedding cache all consume this seam, so a durable, scalable Valkey-backed backend benefits them with zero caller changes.

Why valkey-go and not valkey-glide? The official valkey-glide Go client is built on a Rust core via FFI, which forces CGO and a per-arch native library — that breaks local-store's CGO_ENABLED=0 static build and the Linux/Darwin backend matrix. valkey-go is pure Go with a typed FT.SEARCH builder, a VectorString32 encoder, and a gomock mock package for unit tests.

Testing

Unit tests (valkey-go mock, no container): 35 specs — RPC command-shape/wire contract, empty/len/dim rejects, omit-missing Get, tolerate-missing Delete, topK<1 reject, distance→similarity conversion (0→1, 1→0, 2→−1), lazy FT.CREATE, HNSW arg-shape, key-encoding round-trip (incl. -0.0/NaN), namespace-token collision resistance, config parsing. make test / go run ...ginkgo -r backend/go/valkey-store.
Integration tests (Ginkgo, Label("valkey"), env-gated on VALKEY_ADDR, against valkey/valkey-bundle:9.1.0 on :6379): 14 specs mirroring the local-store suite one-for-one (set/get/delete/find, exact cosine for orthogonal/opposite unit + non-unit vectors, triangle inequality incl. random 768-d) plus a persistence/restart test and a CLIENT LIST client-name assertion. Index back-fill is absorbed with a bounded find poll (no time.Sleep). Skipped automatically when VALKEY_ADDR is unset, so unit CI needs no container.
golangci-lint clean on the changed packages. All Valkey logic lives in backend/go/valkey-store/ (outside COVERAGE_COVERPKG), so the coverage ratchet is unaffected.

Run integration locally:

podman run -d --name valkey-store-it -p 6379:6379 valkey/valkey-bundle:9.1.0
make backends/valkey-store
VALKEY_ADDR=localhost:6379 make test-valkey-store

Scope / follow-ups

Purely additive and opt-in per request. Deferred to follow-ups: hybrid / metadata-filter search (needs new API surface beyond the four RPCs), Valkey Cluster mode, and single-instance multi-namespace consolidation.

Notes for Reviewers

cc @mudler

Signed commits

Yes, I signed my commits.

richiejp · 2026-07-11T10:37:35Z

+# VALKEY_HNSW_M=16
+# VALKEY_HNSW_EF_CONSTRUCTION=200
+# VALKEY_HNSW_EF_RUNTIME=10
+# VALKEY_REQUEST_TIMEOUT_MS=5000


I think at least some of these should go in a model config. Then you can have multiple stores configs.

Done — the backend now reads its config from the store's model config instead of VALKEY_* env vars. A store is configured with a model YAML (name: = the store, backend: valkey-store, and an options: list like addr:host:6379 / index_algo:HNSW), so different stores can each point at their own Valkey server/index within one process. Dropped the env var block from .env and documented the options in stores.md.

richiejp · 2026-07-11T10:42:17Z

+		}
+
+		BeforeEach(func() {
+			valkeyAddr = os.Getenv("VALKEY_ADDR")


Please avoid putting env accesses throughout the code. Model config is the natural place instead of env vars.

Done — removed all env access from the backend. loadConfig now parses the model-config options: list threaded through LoadModel (ModelOptions.Options), and core/backend/stores.go resolves each store's ModelConfig and passes its options down. The integration test configures the store through those same options; VALKEY_ADDR remains only as the harness gate that locates the test server, not as backend config.

daric93 · 2026-07-15T01:09:38Z

@mudler This PR is ready for a second set of eyes and merge whenever you get a chance 🙏

richiejp · 2026-07-15T13:52:13Z

+### This backend is configured through a model config named after the store, not
+### environment variables — create a YAML with `backend: valkey-store` and an
+### `options:` list (addr, index_algo, distance_metric, ...). Different stores can
+### then use different Valkey servers/indexes. See docs/content/features/stores.md.


We don't need this

Done — removed the comment block. The configuration is already documented in docs/content/features/stores.md.

richiejp

Thanks!

localai-bot · 2026-07-16T14:47:44Z

Heads up @daric93: this branch now conflicts with master in three files, all of the append-both-sides kind:

.github/backend-matrix.yml
Makefile
backend/index.yaml

New backends landed on master since this was opened, so the resolution is a union: keep master's new entries AND the valkey-store ones (in backend/index.yaml keep both blocks; same for the matrix entry and the Makefile target). A rebase or a merge of master should take a couple of minutes.

Once that's done this is unblocked: it already has an approval from richiejp and CI was green. @mudler good to merge after the rebase.

localai-bot · 2026-07-16T16:15:24Z

@mudler recommendation: merge this one. Between the two competing implementations of #10708 (this and #10801), this PR is the more complete: it has richiejp's approval after three addressed review rounds, a full green CI run (including both cpu-valkey-store image builds and the darwin metal build), mock-based unit tests that don't need a container in CI, an env-gated integration suite mirroring local-store's, and it is the only one where per-store model config actually works end to end (the core/backend/stores.go threading).

Three things before merge:

DCO is still red: one commit (07d05fa, the .env comment removal) is missing Signed-off-by. A signoff rebase of that commit or a DCO remediation commit fixes it.
Merge conflicts with master in .github/backend-matrix.yml, Makefile, and backend/index.yaml: all append-both-sides, so the resolution is a simple union.
One checklist item from .agents/adding-backends.md worth adding (pre-merge or as an immediate follow-up): Load doesn't gate on the store.NamespacePrefix marker, so if a Valkey server happens to be reachable at the default localhost:6379, the model loader's greedy autoload probe could bind an arbitrary model name to valkey-store (the If you add opus as a backend via api, then add a model via the api that should use llama.cpp or vllm, it then tries to use opus #9287 failure mode). local-store's Load shows the pattern, and feat(stores): add valkey-store vector store backend #10801 implements exactly this gate with a test, so it's a ~5-line addition.

As a nice follow-up (no blocker), #10801's username_env/password_env credential indirection would avoid putting the Valkey password in the model YAML. Great work @daric93, especially the FT.INFO dimension recovery after restart and the documented pipelined-write deadlock finding.

Add a new built-in Go gRPC store backend 'valkey-store' that implements the four Stores RPCs (Set/Get/Delete/Find) against the Valkey Search module (FT.*) using the pure-Go github.com/valkey-io/valkey-go client. It is selected via the existing per-request 'backend' field on /stores, so there is no proto or HTTP API change, and it mirrors the in-memory local-store while adding persistence across restarts and opt-in HNSW. Each vector is a Valkey HASH keyed by hex(little-endian float32); the index is created lazily on first Set (FLAT+COSINE by default), cosine similarity is derived as 1-distance, and namespaces get a collision-resistant token. Includes unit tests (valkey-go mock) and env-gated integration tests against valkey/valkey-bundle, plus build/matrix/gallery wiring and docs. Assisted-by: Kiro:claude-opus-4.8 golangci-lint Signed-off-by: Daria Korenieva <daric2612@gmail.com>

- Load now recovers the persisted vector DIM from FT.INFO (not just index existence), so a post-restart Set/Find validates against the real DIM instead of silently re-learning a wrong one and dropping mismatched vectors from the index. This also restores Find's dimension check after a restart. - StoresFind treats a dropped/missing index as an empty store (empty result, no error) and clears the stale indexCreated flag, matching local-store's empty-store behaviour. - StoresSet reuses checkDims for its per-key length check so the four RPCs share one dimension-guard implementation. - Add unit tests for FT.INFO dimension recovery, loadIndexState, and the dropped-index Find path. Assisted-by: Kiro:claude-opus-4.8 Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…il-fast Addresses external review comments on the valkey-store backend: - StoresFind now rejects a nil/empty query Key before dereferencing it, so a malformed gRPC request can no longer panic the backend. - TLS: derive ServerName (SNI) from the VALKEY_ADDR host so certificate verification works for IP-addressed endpoints, and add VALKEY_TLS_CA_CERT (custom CA bundle) and VALKEY_TLS_SKIP_VERIFY (testing-only) knobs. - Config integer parsing now fails fast on a malformed value (e.g. VALKEY_HNSW_M=1x6) instead of silently defaulting, matching the fail-fast behaviour of the index-algo/distance-metric validation. - Add VALKEY_DB (SELECT n) support for logical-DB isolation. - Cap the human-readable part of a namespace token at 64 chars so a very long model name cannot produce an unbounded key prefix / index name (the appended short hash keeps distinct namespaces collision-free). - Document the KNN-query injection-safety invariant (fields are constants) and why StoresGet uses a single aggregate DoMulti deadline for reads. - Unit tests for the Find nil/empty-key guard, fail-fast HNSW parsing, and VALKEY_DB parsing/validation; docs + .env updated for the new vars. Assisted-by: Kiro:claude-opus-4.8 golangci-lint Signed-off-by: Daria Korenieva <daric2612@gmail.com>

richiejp asked that the valkey-store backend take its configuration from a model config rather than process-wide VALKEY_* environment variables, so multiple stores can each have their own Valkey config within one LocalAI process. This removes every env access from the backend and routes config through the model-config seam every other backend uses. - config.go: loadConfig(opts *pb.ModelOptions) now parses the model config `options:` list (key:value strings, split on the first ':') instead of os.Getenv. Option keys mirror the old VALKEY_* names without the prefix (addr, index_algo, distance_metric, ...). Defaults, fail-fast validation and the mandatory client name are unchanged. - store.go: Load threads opts into loadConfig; TLS comments/errors renamed off the VALKEY_* names. - core/backend/stores.go: StoreBackend and NewVectorStore take a *config.ModelConfigLoader, resolve the per-store ModelConfig by store name, and pass its Options (and Backend when unset) to the backend via WithLoadGRPCLoadModelOpts. No config -> default backend + built-in defaults, preserving the zero-config experience. - Endpoints/routes/application: thread the config loader to StoreBackend. - Unit + integration tests: configure via options; the integration test passes addr through the model-config path (VALKEY_ADDR is now only the test harness locating the server). - docs + .env: document the model-config options, drop the env var table. Assisted-by: Kiro:claude-opus-4.8 Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…nfigured via model config, not env vars — the comment was unnecessary noise in .env. The configuration is already documented in docs/content/features/stores.md. Signed-off-by: Daria Korenieva <daric2612@gmail.com>

…robing Mirror local-store's pattern: reject model names without store.NamespacePrefix so the model loader's greedy autoload probe cannot bind an arbitrary model name to the vector store backend (the mudler#9287 failure mode). Also adds unit tests for the gate covering: prefixed namespace, prefix alone, unprefixed model name, empty model, and nil opts. Signed-off-by: Daria Korenieva <daric2612@gmail.com>

daric93 · 2026-07-16T18:43:05Z

All three items addressed:

DCO — rebased with --signoff on all 6 commits. The missing signoff on 07d05fa is now fixed (4db5b1da).
Merge conflicts — rebased onto current master; the append-both-sides conflicts in backend-matrix.yml, Makefile, and backend/index.yaml are resolved (union of both entries).
NamespacePrefix gate — added in b7d7287e. Load now refuses model names without store.NamespacePrefix (same pattern as local-store), with 5 unit tests covering prefixed namespaces, the prefix alone, unprefixed model names, empty model, and nil opts.

CI should be green again — all valkey-store unit tests pass locally (54/54).

…ion Add support for resolving Valkey credentials from environment variables named in the model config, mirroring cloud-proxy's api_key_env pattern. This keeps secrets out of model YAML files and lets distinct store configs each reference their own credentials. Options: username_env / password_env name the env var holding the value. The direct username / password options still work and take precedence when both are set (backward compatible). Includes 5 unit tests and updated stores.md documentation. Signed-off-by: Daria Korenieva <daric2612@gmail.com>

daric93 · 2026-07-16T18:53:02Z

Also added the username_env/password_env credential indirection from #10801's pattern in aadaccb8 — model configs can now reference env vars holding the credentials instead of putting secrets directly in the YAML:

options:
  - username_env:MY_VALKEY_USER
  - password_env:MY_VALKEY_PASSWORD

Direct username/password options still work and take precedence (backward compatible). Docs updated in stores.md.

localai-bot · 2026-07-16T19:10:51Z

Re-reviewed the current head after the rebase and the follow-up commits. The three earlier blockers are confirmed addressed: DCO is green (all commits signed off), the store.NamespacePrefix gate on Load is in with tests, and the username_env/password_env credential indirection is wired and documented. The end-to-end config threading through core/backend/stores.go (per-store model config resolving backend: + options:) is the piece that makes model-config settings actually reach the backend, and it is unique to this PR.

However, the conflict resolution in the rebase went wrong in two files, and since CI has not run on this head yet (fork awaiting approval, only the DCO check executed), nothing has caught it:

.github/backend-matrix.yml breaks the cloud-proxy backend. In include:, the two valkey-store entries were merged INTO the existing cloud-proxy entries instead of being added as new list items. Both cloud-proxy Linux entries now carry duplicated tag-suffix / runs-on / base-image / skip-drivers / backend keys with the valkey-store values second (see around line 5019: tag-suffix: '-cpu-cloud-proxy' followed a few lines later by tag-suffix: '-cpu-valkey-store' inside the same item). Last-key-wins turns both cloud-proxy Linux builds into valkey-store builds, so cloud-proxy Linux images silently stop being built, and yamllint's duplicate-key check should fail the Yamllint job anyway. In includeDarwin: the cloud-proxy entry also lost its build-type: "metal" and lang: "go" lines to the new valkey-store entry. Fix: make the valkey-store entries standalone list items and restore the cloud-proxy entries exactly as on master.
Makefile has duplicated aggregate lines. There are now two .NOTPARALLEL: lines (2 and 3) and two docker-build-backends: target lines (1411 and 1412); the PR's copies are the pre-rebase versions (missing master's bonsai, cloud-proxy, longcat-video entries) with valkey-store appended. Make accumulates prerequisites so this happens to union out, but please collapse each back to a single line that is master's current line plus the valkey-store additions.

Neither issue touches the backend code itself, which I have no further findings on: the mock-based unit suite (Ginkgo, no container in CI), the env-gated integration suite mirroring local-store plus the persistence and CLIENT LIST specs, the FT.INFO dimension recovery, and the documented pipelined-write deadlock workaround are all solid.

Three small optional pickups that #10801 has and this PR does not: a valkey-store row in docs/content/reference/compatibility-table.md, a line in backend/README.md, and a /valkey-store entry in .gitignore (the binary is built at the repo-relative backend dir, so this one mostly matters for local builds).

@mudler recommendation: this PR and #10801 are competing implementations of #10708. I recommend merging this one once the two rebase artifacts above are fixed and CI is approved to run on the head. It was opened first by the issue author, has been through three review rounds with @richiejp (his changes-requested was dismissed with thanks after the fixes), and is the only one of the two where per-store model config works end to end. #10801 should then be closed with credit: its namespace gate and env-indirect credentials arrived there first and have already been adopted here.

…two issues introduced by the conflict-resolution script during the rebase onto master: 1. .github/backend-matrix.yml: valkey-store entries were merged INTO the cloud-proxy entries (duplicate keys in same YAML map items) instead of being separate list items. This broke cloud-proxy Linux builds and the cloud-proxy darwin entry lost its build-type/lang. Fixed by making them standalone entries and restoring cloud-proxy exactly as on master. 2. Makefile: duplicated .NOTPARALLEL and docker-build-backends lines. Collapsed to single lines that are master's current content plus the valkey-store additions. Also adds the three optional pickups from mudler#10801: - /valkey-store in .gitignore (the built binary) - valkey-store row in docs/content/reference/compatibility-table.md - valkey-store line in backend/README.md Signed-off-by: Daria Korenieva <daric2612@gmail.com>

daric93 · 2026-07-16T20:35:52Z

Fixed both rebase artifacts in 58d1709a:

backend-matrix.yml — cloud-proxy and valkey-store are now separate list items with no duplicate keys. Cloud-proxy's darwin entry has its build-type: "metal" and lang: "go" restored.
Makefile — collapsed to single .NOTPARALLEL: and docker-build-backends: lines that are master's current content + valkey-store.

Also picked up the three optional items: /valkey-store in .gitignore, the compatibility table row, and the backend/README.md line.

CI should be clean now.

localai-bot · 2026-07-17T12:06:06Z

Follow-up after the merge from master (head b59bbce28): both rebase artifacts I flagged are now resolved and the full check suite is green.

.github/backend-matrix.yml: cloud-proxy and valkey-store are now distinct list items in both include: and includeDarwin: (no more merged/duplicated keys), and the darwin cloud-proxy entry kept its build-type: metal / lang: go. Consistent with Yamllint now passing.
Makefile: single .NOTPARALLEL: and single docker-build-backends: lines, each carrying master's full set (incl. bonsai, cloud-proxy, longcat-video) plus the valkey-store additions.

No further findings; the backend code was already clean in my earlier rounds.

@mudler this is good to merge. It and #10801 are competing implementations of #10708; my recommendation stands: merge this one (opened first by the issue author, three review rounds, per-store model config works end to end) and close #10801 with credit for the namespace gate + env-indirect credentials that were adopted here.

daric93 force-pushed the feature/valkey-store-backend branch from f72d11f to 17c3e0a Compare July 10, 2026 20:26

daric93 mentioned this pull request Jul 10, 2026

feat: add Valkey Search vector store backend daric93/LocalAI#1

Closed

daric93 marked this pull request as ready for review July 10, 2026 20:28

daric93 force-pushed the feature/valkey-store-backend branch from 17c3e0a to 86ac171 Compare July 10, 2026 20:33

mudler requested a review from richiejp July 11, 2026 07:19

mudler added the needs-review label Jul 11, 2026

richiejp reviewed Jul 11, 2026

View reviewed changes

daric93 requested a review from richiejp July 13, 2026 20:50

richiejp approved these changes Jul 14, 2026

View reviewed changes

richiejp requested changes Jul 15, 2026

View reviewed changes

daric93 requested a review from richiejp July 15, 2026 20:20

richiejp previously approved these changes Jul 16, 2026

View reviewed changes

localai-bot mentioned this pull request Jul 16, 2026

feat(stores): add valkey-store vector store backend #10801

Open

1 task

daric93 added 6 commits July 16, 2026 11:25

daric93 dismissed richiejp’s stale review via b7d7287 July 16, 2026 18:42

daric93 force-pushed the feature/valkey-store-backend branch from 07d05fa to b7d7287 Compare July 16, 2026 18:42

daric93 requested a review from richiejp July 16, 2026 18:53

Merge branch 'master' into feature/valkey-store-backend

b59bbce

Uh oh!

Conversation

daric93 commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richiejp Jul 11, 2026

Choose a reason for hiding this comment

Uh oh!

daric93 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

richiejp Jul 11, 2026

Choose a reason for hiding this comment

Uh oh!

daric93 Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

daric93 commented Jul 15, 2026

Uh oh!

richiejp Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

daric93 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

richiejp left a comment

Choose a reason for hiding this comment

Uh oh!

localai-bot commented Jul 16, 2026

Uh oh!

localai-bot commented Jul 16, 2026

Uh oh!

daric93 commented Jul 16, 2026

Uh oh!

daric93 commented Jul 16, 2026

Uh oh!

localai-bot commented Jul 16, 2026

Uh oh!

daric93 commented Jul 16, 2026

Uh oh!

localai-bot commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

daric93 commented Jul 10, 2026 •

edited

Loading