Bug: torch 2.8 meta tensor crash on MPS — SentenceTransformer falls back to MockEmbedder

[bradhallett]

Description

On Apple Silicon with PyTorch 2.8, SentenceTransformer instantiation with device=None causes a meta tensor crash:

Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to()

This causes both memora/embedder.py and msa/encoder.py to silently fall back to MockEmbedder, producing near-zero similarity scores and broken retrieval.

Affected Files

• memora/embedder.py (line ~52)
• msa/encoder.py (line ~71)

Environment

• macOS arm64 (Apple Silicon)
• PyTorch 2.8.0
• Python 3.9
• sentence-transformers (latest)

Root Cause

The model config for certain architectures (e.g., jina-embeddings-v3, nomic with remote code) uses torch_dtype which triggers a deprecation path in torch 2.8. When device=None, the auto-resolution logic tries to call .to() on meta tensors, which fails.

Fix

Explicitly set device="mps" on Apple Silicon with a PYTORCH_MPS_ENABLED=0 env override for CPU fallback:

def _load(self):
    if self._model is None:
        import os
        from sentence_transformers import SentenceTransformer
        device = "cpu" if os.environ.get("PYTORCH_MPS_ENABLED") == "0" else "mps"
        self._model = SentenceTransformer(self.model_name, trust_remote_code=True, device=device)
        logger.info("Loaded SentenceTransformer: %s on %s", self.model_name, device)

This matches the pattern already used in memory_server.py and reranker.py.

Secondary Issue: Config YAML overrides silently ignored

The Python config defaults in memora/config.py and msa/config.py can be silently overridden by memora/config.yaml and msa/config.yaml. Changing the Python defaults alone has no effect, which makes model swaps appear to fail. A note in the README or a config validation warning would help. (Happy to file this as a separate issue if preferred.)

Proposal

I also have a scripts/reembed_corpus.py utility for migrating vector stores when switching embedding models — happy to include that in the PR if it would be useful.

Happy to open a PR for both fixes.