Skip to content

Commit 037438e

Browse files
echarlaixecharlaix
authored
Add transformers v5.5 support (#1684)
* add transformers v5.5 support * set min required transformers version to v4.57 * fix position ids generation * fix position_ids generation for qwenvl models * create attention_mask when needed as not created in generate since v5.2 * config loading failing coming from type mismatch coming from transformers v5.4 * filter architectures in tests depending on supported transformers version * deprecate OVModelForVision2Seq * update documentation * remove gemma4 and from qwen3.5 from preview model workflow * set MIN/MAX TRANSFORMERS_VERSION for vlm configs * add legacy remote code models workflow * fix qwen3_omni_moe test expected int8 * pin sentencepiece for transformers 4.56.7
1 parent 6898d97 commit 037438e

28 files changed

Lines changed: 1326 additions & 3128 deletions

‎.github/workflows/test_openvino.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ jobs:
3939
"*quantization*",
4040
"*transformation*",
4141
]
42-
transformers-version: ["4.45.0", "4.57.6", "latest"]
42+
transformers-version: ["4.57.6", "latest"]
4343

4444
runs-on: ubuntu-22.04
4545

@@ -75,12 +75,12 @@ jobs:
7575
run: |
7676
uv pip install qwen-asr
7777
78-
- if: ${{ matrix.transformers-version == '4.45.0' }}
79-
name: Install specific dependencies and versions required for older transformers
78+
- if: ${{ matrix.transformers-version == '4.57.6' }}
79+
name: Pin sentencepiece (0.2.2 fails to load some legacy remote-code tokenizer.model files)
8080
run: |
81-
uv pip install transformers==${{ matrix.transformers-version }} accelerate==0.* peft==0.13.* diffusers==0.32.* transformers_stream_generator
81+
uv pip install "sentencepiece<0.2.2"
8282
83-
- if: ${{ matrix.transformers-version != '4.45.0' && matrix.test-pattern == '*decoder*'}}
83+
- if: ${{ matrix.test-pattern == '*decoder*'}}
8484
name: Install auto-gptq, autoawq
8585
run: |
8686
uv pip install auto-gptq "autoawq<0.2.8"

‎.github/workflows/test_openvino_legacy_remote_code_models.yml‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
name: Legacy Remote Code Models Validation
2+
3+
on:
4+
workflow_dispatch:
5+
push:
6+
branches:
7+
- main
8+
- v*-release
9+
pull_request:
10+
branches:
11+
- main
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
15+
cancel-in-progress: true
16+
17+
permissions:
18+
contents: read
19+
20+
env:
21+
UV_TORCH_BACKEND: cpu
22+
UV_SYSTEM_PYTHON: true
23+
TRANSFORMERS_IS_CI: true
24+
HF_TOKEN: ${{ secrets.HF_HUB_READ_TOKEN }}
25+
26+
jobs:
27+
build:
28+
strategy:
29+
fail-fast: false
30+
matrix:
31+
test-pattern:
32+
[
33+
"*decoder*",
34+
"*seq2seq*",
35+
]
36+
37+
runs-on: ubuntu-22.04
38+
39+
steps:
40+
- name: Checkout code
41+
uses: actions/checkout@v7.0.0
42+
43+
- name: Setup Python
44+
uses: actions/setup-python@v6
45+
with:
46+
python-version: "3.10"
47+
48+
- name: Install dependencies
49+
run: |
50+
pip install --upgrade pip uv
51+
uv pip install .[tests] librosa diffusers
52+
53+
- name: Login with fork PRs CI token
54+
if: ${{ env.HF_TOKEN == '' }}
55+
run: |
56+
python tests/scripts/login_with_ci_token.py
57+
58+
- name: Legacy remote-code models validation
59+
run: |
60+
uv pip install "transformers==4.51.0" transformers_stream_generator
61+
pytest tests/openvino/${{ matrix.test-pattern }} -k "(chatglm and not chatglm4) or (minicpm and not minicpmv) or (qwen and not qwen2 and not qwen3 and not videochat_flash_qwen) or arctic or deepseek or llava-qwen2 or phi3_v or phi4mm or minicpmo or test_filtered_architectures" --durations=0

‎.github/workflows/test_openvino_preview_models.yml‎

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,6 @@ jobs:
3232
[
3333
"*export*",
3434
"*seq2seq*",
35-
"*quantization*",
3635
"*transformation*",
3736
]
3837

@@ -61,18 +60,7 @@ jobs:
6160
run: |
6261
uv pip install --pre -U openvino openvino-tokenizers openvino-genai --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
6362
64-
- name: Gemma 4 Validation
65-
run: |
66-
uv pip install transformers==5.5.0
67-
pytest tests/openvino/${{ matrix.test-pattern }} -k "gemma4 and not gemma4_unified" --durations=0
68-
6963
- name: Gemma 4 Unified Validation
70-
if: ${{ matrix.test-pattern != '*quantization*' }}
7164
run: |
7265
uv pip install "transformers>=5.10,<5.11"
73-
pytest tests/openvino/${{ matrix.test-pattern }} -k gemma4_unified --durations=0
74-
75-
- name: Qwen3.5 and Qwen3.5-Moe Validation
76-
run: |
77-
uv pip install "transformers>=5.2,<5.3"
78-
pytest tests/openvino/${{ matrix.test-pattern }} -k "qwen3_5 or qwen3_5_moe" --durations=0
66+
pytest tests/openvino/${{ matrix.test-pattern }} -k gemma4_unified --durations=0

‎.github/workflows/test_openvino_slow.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ jobs:
3636
fail-fast: false
3737
matrix:
3838
os: ["ubuntu-22.04", "windows-2022"]
39-
transformers-version: ["4.45.0", "latest"]
39+
transformers-version: ["4.57.6", "latest"]
4040
include:
4141
- transformers-version: "main"
4242
os: "ubuntu-22.04"

‎docs/source/openvino/inference.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ As shown in the table below, each task is associated with a class enabling to au
4848
| `OVModelForCausalLM` | `text-generation-with-past` |
4949
| `OVModelForSeq2SeqLM` | `text2text-generation-with-past` |
5050
| `OVModelForSpeechSeq2Seq` | `automatic-speech-recognition` |
51-
| `OVModelForVision2Seq` | `image-to-text` |
51+
| `OVModelForImageTextToText` | `image-to-text` |
5252
| `OVModelForTextToSpeechSeq2Seq` | `text-to-audio` |
5353

5454
### Diffusers models

‎docs/source/openvino/reference.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,9 @@ The following classes are available for the following computer vision tasks.
102102

103103
The following classes are available for the following multimodal tasks.
104104

105-
### OVModelForVision2Seq
105+
### OVModelForImageTextToText
106106

107-
[[autodoc]] openvino.modeling_seq2seq.OVModelForVision2Seq
107+
[[autodoc]] openvino.modeling_seq2seq.OVModelForImageTextToText
108108
- forward
109109

110110
### OVModelForPix2Struct

‎optimum/exporters/openvino/base.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ class OpenVINOConfig(ExporterConfig, ABC):
7171
DEFAULT_VARIANT = "default"
7272
PATCHING_SPECS: list[PatchingSpec] | None = None
7373
_MODEL_PATCHER = ModelPatcher
74+
MIN_TRANSFORMERS_VERSION = "4.57"
7475

7576
_TASK_TO_COMMON_OUTPUTS = { # noqa: RUF012
7677
"audio-classification": OrderedDict({"logits": {0: "batch_size"}}),

‎optimum/exporters/openvino/input_generators.py‎

Lines changed: 20 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@
3030
FalconDummyPastKeyValuesGenerator,
3131
MistralDummyPastKeyValuesGenerator,
3232
NormalizedTextConfig,
33-
is_transformers_version,
3433
)
3534
from optimum.utils.input_generators import DTYPE_MAPPER
3635
from optimum.utils.normalized_config import NormalizedConfig, NormalizedVisionConfig
@@ -42,46 +41,27 @@ def __init__(self, task: str, normalized_config: NormalizedTextConfig, **kwargs)
4241
self.multi_query = normalized_config.multi_query
4342

4443
def generate(self, input_name: str, framework: str = "pt", int_dtype: str = "int64", float_dtype: str = "fp32"):
45-
if is_transformers_version("<", "4.54"):
46-
if self.multi_query:
47-
shape = (
48-
self.batch_size,
49-
self.sequence_length,
50-
self.hidden_size // self.num_attention_heads * 2,
51-
)
52-
else:
53-
shape = (
54-
self.batch_size,
55-
self.num_attention_heads,
56-
self.sequence_length,
57-
self.hidden_size // self.num_attention_heads * 2,
58-
)
59-
pkv = [
60-
self.random_float_tensor(shape, framework=framework, dtype=float_dtype) for _ in range(self.num_layers)
61-
]
62-
44+
if self.multi_query:
45+
shape = (
46+
self.batch_size,
47+
1,
48+
self.sequence_length,
49+
self.hidden_size // self.num_attention_heads,
50+
)
6351
else:
64-
if self.multi_query:
65-
shape = (
66-
self.batch_size,
67-
1,
68-
self.sequence_length,
69-
self.hidden_size // self.num_attention_heads,
70-
)
71-
else:
72-
shape = (
73-
self.batch_size,
74-
self.num_attention_heads,
75-
self.sequence_length,
76-
self.hidden_size // self.num_attention_heads,
77-
)
78-
pkv = [
79-
(
80-
self.random_float_tensor(shape, framework=framework, dtype=float_dtype),
81-
self.random_float_tensor(shape, framework=framework, dtype=float_dtype),
82-
)
83-
for _ in range(self.num_layers)
84-
]
52+
shape = (
53+
self.batch_size,
54+
self.num_attention_heads,
55+
self.sequence_length,
56+
self.hidden_size // self.num_attention_heads,
57+
)
58+
pkv = [
59+
(
60+
self.random_float_tensor(shape, framework=framework, dtype=float_dtype),
61+
self.random_float_tensor(shape, framework=framework, dtype=float_dtype),
62+
)
63+
for _ in range(self.num_layers)
64+
]
8565

8666
return pkv
8767

0 commit comments

Comments
 (0)