Skip to content

Commit 036340b

Browse files
XingweiDengzhangyue66vasqu
authored
[Model] Add PP-OCRV5_mobile_det Model Support (huggingface#43247)
* Feat: Add PP-OCRV5_mobile_det model * fix code * fix code * fix * use cv and np to replace pyclipper * add model post_init() * fix * fix model init_weight * fix rename module * update * update * update * update * update * update * update * update * update * update * update * update * update * Feat: Add PP-LCNet model * fix doc * update * update * update * update * update * update * update * refactor * init pp_lcnet_v3 * use load_backbone * clean * update * fix * add pp_lcnet_v3 docs and tests * fix * update lcnet test * fix * reuse server_det * update * fix * fixup modular * fix * fix docs --------- Co-authored-by: zhangyue66 <zhangyue66@baidu.com> Co-authored-by: vasqu <antonprogamer@gmail.com> Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
1 parent 2548d0d commit 036340b

33 files changed

Lines changed: 3931 additions & 14 deletions

‎docs/source/en/_toctree.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1268,8 +1268,14 @@
12681268
title: PP-DocLayoutV2
12691269
- local: model_doc/pp_doclayout_v3
12701270
title: PP-DocLayoutV3
1271+
- local: model_doc/pp_ocrv5_mobile_det
1272+
title: PP-OCRv5_mobile_det
12711273
- local: model_doc/pp_ocrv5_server_det
12721274
title: PP-OCRv5_server_det
1275+
- local: model_doc/pp_lcnet
1276+
title: PPLCNet
1277+
- local: model_doc/pp_lcnet_v3
1278+
title: PPLCNetV3
12731279
- local: model_doc/qwen2_5_omni
12741280
title: Qwen2.5-Omni
12751281
- local: model_doc/qwen2_5_vl

‎docs/source/en/model_doc/pp_lcnet.md‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
<!--Copyright 2026 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
12+
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
13+
rendered properly in your Markdown viewer.
14+
15+
-->
16+
*This model was released on 2021-09-17 and added to Hugging Face Transformers on 2026-03-13.*
17+
18+
# PP-LCNet
19+
20+
<div class="flex flex-wrap space-x-1">
21+
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
22+
</div>
23+
24+
## Overview
25+
26+
**PP-LCNet** PP-LCNet is a family of efficient, lightweight convolutional neural networks designed for real-world document understanding and OCR tasks. It balances accuracy, speed, and model size, making it ideal for both server-side and edge deployment. To address different document processing requirements, PP-LCNet has three main variants, each optimized for a specific task.
27+
28+
## Model Architecture
29+
30+
1. The Document Image Orientation Classification Module is primarily designed to distinguish the orientation of document images and correct them through post-processing. During processes such as document scanning or ID photo capturing, the device might be rotated to achieve clearer images, resulting in images with various orientations. Standard OCR pipelines may not handle these images effectively. By leveraging image classification techniques, the orientation of documents or IDs containing text regions can be pre-determined and adjusted, thereby improving the accuracy of OCR processing.
31+
32+
2. The Table Classification Module is a key component in computer vision systems, responsible for classifying input table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Classification Module typically receives table images as input and, using deep learning algorithms, classifies them into predefined categories based on the characteristics and content of the images, such as wired and wireless tables. The classification results from the Table Classification Module serve as output for use in table recognition pipelines.
33+
34+
3. The text line orientation classification module primarily distinguishes the orientation of text lines and corrects them using post-processing. In processes such as document scanning and license/certificate photography, to capture clearer images, the capture device may be rotated, resulting in text lines in various orientations. Standard OCR pipelines cannot handle such data well. By utilizing image classification technology, the orientation of text lines can be predetermined and adjusted, thereby enhancing the accuracy of OCR processing.
35+
36+
37+
## Usage
38+
39+
### Single input inference
40+
41+
The example below demonstrates how to classify image with PP-LCNet using [`Pipeline`] or the [`AutoModel`].
42+
43+
<hfoptions id="usage">
44+
<hfoption id="Pipeline">
45+
46+
```py
47+
import requests
48+
from PIL import Image
49+
from transformers import pipeline
50+
51+
model_path = "PaddlePaddle/PP-LCNet_x1_0_doc_ori_safetensors"
52+
image_classifier = pipeline("image-classification", model=model_path, function_to_apply="none", device_map="auto")
53+
54+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg", stream=True).raw)
55+
result = image_classifier(image)
56+
print(result)
57+
```
58+
59+
</hfoption>
60+
61+
<hfoption id="AutoModel">
62+
63+
```py
64+
import requests
65+
from PIL import Image
66+
from transformers import AutoImageProcessor, AutoModelForImageClassification
67+
68+
model_path = "PaddlePaddle/PP-LCNet_x1_0_doc_ori_safetensors"
69+
model = AutoModelForImageClassification.from_pretrained(model_path, device_map="auto")
70+
image_processor = AutoImageProcessor.from_pretrained(model_path)
71+
72+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg", stream=True).raw)
73+
inputs = image_processor(images=image, return_tensors="pt").to(model.device)
74+
outputs = model(**inputs)
75+
predicted_label = outputs.logits.argmax(-1).item()
76+
print(model.config.id2label[predicted_label])
77+
```
78+
79+
</hfoption>
80+
</hfoptions>
81+
82+
### Batched inference
83+
84+
Here is how you can do it with PP-LCNet using [`Pipeline`] or the [`AutoModel`]:
85+
86+
<hfoptions id="usage">
87+
<hfoption id="Pipeline">
88+
89+
```py
90+
import requests
91+
from PIL import Image
92+
from transformers import pipeline
93+
94+
model_path = "PaddlePaddle/PP-LCNet_x1_0_doc_ori_safetensors"
95+
image_classifier = pipeline("image-classification", model=model_path, function_to_apply="none", device_map="auto")
96+
97+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg", stream=True).raw)
98+
result = image_classifier([image, image])
99+
print(result)
100+
```
101+
102+
</hfoption>
103+
104+
<hfoption id="AutoModel">
105+
106+
```py
107+
import requests
108+
from PIL import Image
109+
from transformers import AutoImageProcessor, AutoModelForImageClassification
110+
111+
model_path = "PaddlePaddle/PP-LCNet_x1_0_doc_ori_safetensors"
112+
model = AutoModelForImageClassification.from_pretrained(model_path, device_map="auto")
113+
image_processor = AutoImageProcessor.from_pretrained(model_path)
114+
115+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg", stream=True).raw)
116+
inputs = image_processor(images=[image, image], return_tensors="pt").to(model.device)
117+
outputs = model(**inputs)
118+
119+
predicted_labels = outputs.logits.argmax(-1)
120+
121+
for label_id in predicted_labels:
122+
label_id_scalar = label_id.item()
123+
label = model.config.id2label[label_id_scalar]
124+
print(label)
125+
```
126+
127+
</hfoption>
128+
</hfoptions>
129+
130+
## PPLCNetForImageClassification
131+
132+
[[autodoc]] PPLCNetForImageClassification
133+
134+
## PPLCNetConfig
135+
136+
[[autodoc]] PPLCNetConfig
137+
138+
## PPLCNetBackbone
139+
140+
[[autodoc]] PPLCNetBackbone
141+
142+
## PPLCNetImageProcessorFast
143+
144+
[[autodoc]] PPLCNetImageProcessorFast

‎docs/source/en/model_doc/pp_lcnet_v3.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
<!--Copyright 2026 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
12+
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
13+
rendered properly in your Markdown viewer.
14+
15+
-->
16+
*This model was released on 2023-04-14 and added to Hugging Face Transformers on 2026-03-13.*
17+
18+
# PP-LCNetV3
19+
20+
<div class="flex flex-wrap space-x-1">
21+
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
22+
</div>
23+
24+
## Overview
25+
26+
PPLCNetV3 is provided as a backbone network only. A pre-trained model for direct use in image classification has not been officially released.
27+
28+
## PPLCNetV3Backbone
29+
30+
[[autodoc]] PPLCNetV3Backbone
31+
32+
## PPLCNetV3Config
33+
34+
[[autodoc]] PPLCNetV3Config

‎docs/source/en/model_doc/pp_ocrv5_mobile_det.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
<!--Copyright 2026 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
12+
⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
13+
rendered properly in your Markdown viewer.
14+
15+
-->
16+
*This model was released on 2025-05-20 and added to Hugging Face Transformers on 2026-03-13.*
17+
18+
# PP-OCRv5_mobile_det
19+
20+
<div class="flex flex-wrap space-x-1">
21+
<img alt="PyTorch" src="https://img.shields.io/badge/PyTorch-DE3412?style=flat&logo=pytorch&logoColor=white">
22+
</div>
23+
24+
## Overview
25+
26+
**PP-OCRv5_mobile_det** is a dedicated lightweight model for text detection, focusing specifically on efficient detection and understanding of text elements in multi-language documents and natural scenes.
27+
28+
## Model Architecture
29+
30+
PP-OCRv5_mobile_det is one of the PP-OCRv5_det series, the latest generation of text detection models developed by the PaddleOCR team. It aims to efficiently and accurately supports the detection of text in diverse scenarios—including handwriting, vertical, rotated, and curved text—across multiple languages such as Simplified Chinese, Traditional Chinese, English, and Japanese. Key features include robust handling of complex layouts, varying text sizes, and challenging backgrounds, making it suitable for practical applications like document analysis, license plate recognition, and scene text detection.
31+
32+
33+
## Usage
34+
35+
### Single input inference
36+
37+
The example below demonstrates how to detect text with PP-OCRV5_Mobile_Det using the [`Pipeline`] or the [`AutoModel`].
38+
39+
<hfoptions id="usage">
40+
<hfoption id="Pipeline">
41+
42+
```py
43+
import requests
44+
from PIL import Image
45+
from transformers import pipeline
46+
47+
image = Image.open(
48+
requests.get(
49+
"https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png", stream=True
50+
).raw)
51+
detector = pipeline(
52+
task="object-detection",
53+
model="PaddlePaddle/PP-OCRV5_mobile_det_safetensors",
54+
device_map="auto",
55+
)
56+
results = detector(image)
57+
58+
for result in results:
59+
print(result)
60+
61+
```
62+
63+
</hfoption>
64+
<hfoption id="AutoModel">
65+
66+
```py
67+
import requests
68+
from PIL import Image
69+
from transformers import AutoImageProcessor, AutoModelForObjectDetection
70+
71+
model_path="PaddlePaddle/PP-OCRv5_mobile_det_safetensors"
72+
model = AutoModelForObjectDetection.from_pretrained(model_path, device_map="auto")
73+
image_processor = AutoImageProcessor.from_pretrained(model_path)
74+
75+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png", stream=True).raw).convert("RGB")
76+
inputs = image_processor(images=image, return_tensors="pt").to(model.device)
77+
outputs = model(**inputs)
78+
79+
results = image_processor.post_process_object_detection(outputs, target_sizes=inputs["target_sizes"])
80+
81+
for result in results:
82+
print(result["boxes"])
83+
print(result["scores"])
84+
85+
```
86+
87+
</hfoption>
88+
</hfoptions>
89+
90+
### Batched inference
91+
92+
Here is how you can do it with PP-OCRV5_Mobile_Det using the [`Pipeline`] or the [`AutoModel`]:
93+
94+
<hfoptions id="usage">
95+
<hfoption id="Pipeline">
96+
97+
```py
98+
import requests
99+
from PIL import Image
100+
from transformers import pipeline
101+
102+
image = Image.open(
103+
requests.get(
104+
"https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png", stream=True
105+
).raw)
106+
detector = pipeline(
107+
task="object-detection",
108+
model="PaddlePaddle/PP-OCRV5_mobile_det_safetensors",
109+
device_map="auto",
110+
)
111+
results = detector([image, image])
112+
113+
for result in results:
114+
print(result)
115+
116+
```
117+
118+
</hfoption>
119+
120+
<hfoption id="AutoModel">
121+
122+
```py
123+
import requests
124+
from PIL import Image
125+
from transformers import AutoImageProcessor, AutoModelForObjectDetection
126+
127+
model_path="PaddlePaddle/PP-OCRv5_mobile_det_safetensors"
128+
model = AutoModelForObjectDetection.from_pretrained(model_path, device_map="auto")
129+
image_processor = AutoImageProcessor.from_pretrained(model_path)
130+
131+
image = Image.open(requests.get("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png", stream=True).raw).convert("RGB")
132+
inputs = image_processor(images=[image, image], return_tensors="pt").to(model.device)
133+
outputs = model(**inputs)
134+
135+
results = image_processor.post_process_object_detection(outputs, target_sizes=inputs["target_sizes"])
136+
137+
for result in results:
138+
print(result["boxes"])
139+
print(result["scores"])
140+
141+
```
142+
143+
</hfoption>
144+
</hfoptions>
145+
146+
## PPOCRV5MobileDetForObjectDetection
147+
148+
[[autodoc]] PPOCRV5MobileDetForObjectDetection
149+
150+
## PPOCRV5MobileDetConfig
151+
152+
[[autodoc]] PPOCRV5MobileDetConfig
153+
154+
## PPOCRV5MobileDetModel
155+
156+
[[autodoc]] PPOCRV5MobileDetModel

‎src/transformers/models/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -319,6 +319,9 @@
319319
from .pop2piano import *
320320
from .pp_doclayout_v2 import *
321321
from .pp_doclayout_v3 import *
322+
from .pp_lcnet import *
323+
from .pp_lcnet_v3 import *
324+
from .pp_ocrv5_mobile_det import *
322325
from .pp_ocrv5_server_det import *
323326
from .prompt_depth_anything import *
324327
from .prophetnet import *

‎src/transformers/models/auto/configuration_auto.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -357,6 +357,9 @@
357357
("pop2piano", "Pop2PianoConfig"),
358358
("pp_doclayout_v2", "PPDocLayoutV2Config"),
359359
("pp_doclayout_v3", "PPDocLayoutV3Config"),
360+
("pp_lcnet", "PPLCNetConfig"),
361+
("pp_lcnet_v3", "PPLCNetV3Config"),
362+
("pp_ocrv5_mobile_det", "PPOCRV5MobileDetConfig"),
360363
("pp_ocrv5_server_det", "PPOCRV5ServerDetConfig"),
361364
("prompt_depth_anything", "PromptDepthAnythingConfig"),
362365
("prophetnet", "ProphetNetConfig"),
@@ -868,6 +871,9 @@
868871
("pop2piano", "Pop2Piano"),
869872
("pp_doclayout_v2", "PPDocLayoutV2"),
870873
("pp_doclayout_v3", "PPDocLayoutV3"),
874+
("pp_lcnet", "PPLCNet"),
875+
("pp_lcnet_v3", "PPLCNetV3"),
876+
("pp_ocrv5_mobile_det", "PPOCRV5MobileDet"),
871877
("pp_ocrv5_server_det", "PPOCRV5ServerDet"),
872878
("prompt_depth_anything", "PromptDepthAnything"),
873879
("prophetnet", "ProphetNet"),

‎src/transformers/models/auto/image_processing_auto.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,8 @@
172172
("poolformer", ("PoolFormerImageProcessor", "PoolFormerImageProcessorFast")),
173173
("pp_doclayout_v2", (None, "PPDocLayoutV2ImageProcessorFast")),
174174
("pp_doclayout_v3", (None, "PPDocLayoutV3ImageProcessorFast")),
175+
("pp_lcnet", (None, "PPLCNetImageProcessorFast")),
176+
("pp_ocrv5_mobile_det", (None, "PPOCRV5ServerDetImageProcessorFast")),
175177
("pp_ocrv5_server_det", (None, "PPOCRV5ServerDetImageProcessorFast")),
176178
("prompt_depth_anything", ("PromptDepthAnythingImageProcessor", "PromptDepthAnythingImageProcessorFast")),
177179
("pvt", ("PvtImageProcessor", "PvtImageProcessorFast")),

0 commit comments

Comments
 (0)