Skip to content

Can't send translate request to huggingface nllb model #6499

@aicswd

Description

@aicswd

I deployed nllb model from huggingface into seldom-core v2 MLserver,
but I can't make any request on it
`import requests
import json
inference_request ={
"inputs": [{
"name": "args",
"shape": [-1],
"datatype": "BYTES",
"parameters":{
'headers': {"Content-Type": "application/json"},
},
"data": ["This is a test sentence."]
},
{
"name": "array_inputs",
"shape": [-1],
"datatype": "BYTES",
"parameters": {"src_lang":"en", "tgt_lang":"fr"}
}
]

}
requests.post(
"http:///v2/models/mt-nllb-200-distilled-600m/infer", json=inference_request
).json()`

This code results in this error
2025-06-03 11:58:19,151 [mlserver.parallel] ERROR - An error occurred calling method 'predict' from model 'mt-nllb-200-distilled-600m_1'. Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/mlserver/parallel/worker.py", line 136, in _process_request return_value = await method( File "/opt/conda/lib/python3.10/site-packages/mlserver_huggingface/runtime.py", line 48, in predict prediction = self._model(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/text2text_generation.py", line 371, in __call__ return super().__call__(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/text2text_generation.py", line 167, in __call__ result = super().__call__(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/base.py", line 1224, in __call__ outputs = list(final_iterator) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 124, in __next__ item = next(self.iterator) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 124, in __next__ item = next(self.iterator) File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 630, in __next__ data = self._next_data() File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 673, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 52, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/pt_utils.py", line 19, in __getitem__ processed = self.process(item, **self.params) File "/opt/conda/lib/python3.10/site-packages/transformers/pipelines/text2text_generation.py", line 319, in preprocess return self.tokenizer._build_translation_inputs( File "/opt/conda/lib/python3.10/site-packages/transformers/models/nllb/tokenization_nllb_fast.py", line 253, in _build_translation_inputs raise ValueError("Translation requires asrc_langand atgt_langfor this model") ValueError: Translation requires asrc_langand atgt_langfor this model
I can't find any documentation to help me solve this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions