[BUG] compute_patch_embeddings fails with DINOv3 HF model #6366
Description
Describe the problem
The compute_patch_embeddings fails with DINOv3 loaded from HuggingFace with the following error:
TypeError: expected Tensor as element 0 in argument 0, but got BatchFeature
Code to reproduce issue
import transformers
import fiftyone.utils.transformers as fouhft
import fiftyone.zoo as foz
dataset = foz.load_zoo_dataset("quickstart", max_samples=25)
transformers_model = transformers.AutoModel.from_pretrained("facebook/dinov3-vitl16-pretrain-lvd1689m")
model_config = fouhft.FiftyOneTransformerConfig(
{
"model": transformers_model,
"name_or_path":"facebook/dinov3-vitl16-pretrain-lvd1689m",
}
)
model = fouhft.FiftyOneTransformer(model_config)
embeddings = dataset.compute_patch_embeddings(model, 'ground_truth', skip_failures=False)System information
- OS Platform and Distribution (e.g., Linux Ubuntu 22.04): Ubuntu 22.04
- Python version (
python --version): 3.12.11 - FiftyOne version (
fiftyone --version): 1.8.1 - FiftyOne installed from (pip or source): pip
Other info/logs
Traceback (most recent call last):
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/runpy.py", line 198, in _run_module_as_main
return _run_code(code, main_globals, None,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/runpy.py", line 88, in _run_code
exec(code, run_globals)
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 71, in <module>
cli.main()
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 501, in main
run()
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 351, in run_file
runpy.run_path(target, run_name="__main__")
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 310, in run_path
return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 127, in _run_module_code
_run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
File "/home/owe/.vscode/extensions/ms-python.debugpy-2025.10.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 118, in _run_code
exec(code, run_globals)
File "/home/owe/code/fiftyone-datasets/owe_explorer.py", line 108, in <module>
embeddings = dataset.compute_patch_embeddings(model, 'ground_truth', skip_failures=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/core/collections.py", line 3743, in compute_patch_embeddings
return fomo.compute_patch_embeddings(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/core/models.py", line 1619, in compute_patch_embeddings
return _embed_patches_data_loader(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/core/models.py", line 1786, in _embed_patches_data_loader
for sample, patches in pb(zip(samples, data_loader)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/eta/core/utils.py", line 1699, in __next__
val = next(self._iterator)
^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 734, in __next__
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1516, in _next_data
return self._process_data(data, worker_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1551, in _process_data
data.reraise()
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/_utils.py", line 769, in reraise
raise exception
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
~~~~~~~~~~~~^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/utils/torch.py", line 2304, in __getitem__
raise e
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/utils/torch.py", line 2301, in __getitem__
img_patches = self._extract_patches(image_path, patches)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/owe/miniconda3/envs/fiftyone/lib/python3.12/site-packages/fiftyone/utils/torch.py", line 2345, in _extract_patches
img_patches = torch.stack(img_patches, dim=0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: expected Tensor as element 0 in argument 0, but got BatchFeature
Willingness to contribute
The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?
- Yes. I can contribute a fix for this bug independently
- Yes. I would be willing to contribute a fix for this bug with guidance
from the FiftyOne community - No. I cannot contribute a bug fix at this time