Skip to content

fix(model-artifacts): gate inferred artifact materialization by backend#10910

Merged
mudler merged 1 commit into
masterfrom
fix/artifact-inferred-backend-gating
Jul 17, 2026
Merged

fix(model-artifacts): gate inferred artifact materialization by backend#10910
mudler merged 1 commit into
masterfrom
fix/artifact-inferred-backend-gating

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

Problem

Follow-up to #10909 (single-file snapshot resolution). That PR fixed the case where an explicitly-resolved single-file snapshot was handed to a backend as a directory. This PR closes the remaining gap: the loader still infers a managed artifact from any HuggingFace-shaped parameters.model regardless of backend.

PrimaryArtifactSpec materializes a bare HF reference (huggingface://owner/repo/model.gguf, hf://…, or a bare owner/repo) into a snapshot directory .artifacts/huggingface/<key>/snapshot/. That directory is the correct load target for directory-consuming backends (transformers, vLLM, diffusers, …), but a single-file backend such as llama.cpp or whisper is then handed the directory instead of the weight file:

gguf_init_from_reader: failed to read magic
llama_model_load: error loading model: llama_model_loader: failed to load model from
  /models/.artifacts/huggingface/<key>/snapshot

The /import-model importer already guards this with a backend allow-list (managedArtifactBackends), but the loader-side inference in core/config never applied the same gate. So a hand-written YAML or local-ai run huggingface://…/x.gguf with a single-file backend and no files: would break — the exact class of failure that motivated #10909, just via the inference path rather than an explicit artifacts: block.

Fix

  • Move the backend allow-list from core/gallery/importers/helpers.go into core/config as an exported predicate IsManagedArtifactBackend(backend).
  • PrimaryArtifactSpec now gates the inferred cases: only directory-consuming backends may have an artifact inferred from a bare reference. Every other backend stays on the legacy download-to-file path, so the backend receives the weight file itself.
  • An explicit artifacts: block still bypasses the gate — single-file snapshot resolution (fix(model-artifacts): load single-file HF snapshots from the file, not the directory #10909) handles the load path there.
  • The importer now consumes the shared config.IsManagedArtifactBackend, so both paths agree on which backends auto-materialize (no duplicate list).

Tests

  • New core/config/model_artifact_inference_test.go (Ginkgo): a bare HF file/repo reference on llama-cpp stays legacy (managed=false); a reference on transformers still infers a managed artifact; an explicit artifacts: block on llama-cpp stays managed.
  • Written test-first (2 of 4 cases failed before the change, then green).

Verified locally: go test ./core/config/ ./core/gallery/importers/ ./core/services/galleryop/ ./core/application/ pass; make lint (new-from-merge-base) reports 0 issues.

Assisted-by: Claude:opus-4.8 [Claude Code]

🤖 Generated with Claude Code

The managed-artifact materializer stages a HuggingFace snapshot into a
directory (.artifacts/huggingface/<key>/snapshot/). That is the right load
target for directory-consuming backends (transformers, vLLM, diffusers, ...),
but PrimaryArtifactSpec inferred a managed artifact from ANY HuggingFace-shaped
model reference regardless of backend. A single-file backend such as llama.cpp
or whisper was therefore handed the snapshot directory instead of the weight
file and failed to load it.

The /import-model importer already guards this with a backend allow-list
(managedArtifactBackends), but the loader-side inference did not. Move the
allow-list into core/config as IsManagedArtifactBackend and apply it in
PrimaryArtifactSpec: only directory-consuming backends may have an artifact
inferred from a bare reference; every other backend stays on the legacy
download-to-file path. An explicit artifacts: block still bypasses the gate,
where single-file snapshot resolution handles the load path.

The importer now shares the same predicate, so both paths agree on which
backends auto-materialize.

Assisted-by: Claude:opus-4.8 [Claude Code]
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler enabled auto-merge (squash) July 17, 2026 23:04
@mudler
mudler merged commit 2dade4a into master Jul 17, 2026
66 checks passed
@mudler
mudler deleted the fix/artifact-inferred-backend-gating branch July 17, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants