查看中文

What Is MindSpore Lite

MindSpore Lite provides lightweight AI inference acceleration capabilities for different hardware devices, enabling intelligent applications and providing end-to-end solutions for developers. It offers development friendly, efficient, and flexible deployment experiences for algorithm engineers and data scientists, helping the AI software and hardware application ecosystem thrive. In the future, MindSpore Lite will work with the MindSpore AI community to enrich the AI software and hardware application ecosystem.

Example

MindSpore Lite achieves double the inference performance for AIGC, speech algorithms, and CV model inference, and has been deployed in Huawei's flagship smartphones for commercial use. As shown in the figure below, MindSpore Lite supports image style transfer and image segmentation for CV algorithms.

Quick Start

1. Compile

   MindSpore Lite has multiple different hardware backends, including:

   • For service side devices, users can compile dynamic libraries and Python wheel packages by setting compilation options such as MSLITE_ENABLE_CLOUD_INFERENCE for inference of upgrade and CPU hardware. For detailed compilation tutorials, please refer to the official website of MindSpore Lite.

   • For end and edge devices, different dynamic libraries can be compiled through different cross compilation toolchains. For detailed compilation tutorials, please refer to the official website of MindSpore Lite.

2. Model conversion

   MindSpore Lite supports the conversion of models serialized from various AI frameworks such as MindSpore, ONNX, TF, etc. into MindSpore Lite format IR. In order to achieve more efficient model inference, MindSpore Lite supports the conversion of models into .ms format or .mindir format, where:

   • The .mindir model is used for inference on service side devices and is more compatible with the model structure exported by the MindSpore training framework. It is mainly suitable for Ascend cards and X86/Arm architecture CPU hardware. For detailed conversion methods, please refer to the Conversion Tool Tutorial.

   • The .ms model is mainly used for inference of end and edge devices, and is mainly suitable for terminal hardware such as Kirin NPU and Arm architecture CPU. In order to better reduce the size of the model file, the .ms model is serialized and deserialized through protobuffer. For detailed instructions on how to use the conversion tool, please refer to the Conversion Tool

3. Model inference

   MindSpore Lite provides three APIs: Python, C++, and Java, and complete usage cases for the corresponding APIs:

   • Python API Interface Use Case

     .mindir Reasoning Case Based on Python Interface

   • C/C++ Complete Use Cases

     .mindir model based on C/C++ Interface Inference Use Case

     .ms Model Based on C/C++ Interface Reasoning Case

   • Complete Java Use Cases

     .mindir model based on Java interface inference use case

Technical Solution

MindSpore Lite Features

• MindSpore Lite exports .mindir models for cloud-side inference on Atlas 300I Duo, Atlas 800IA3 Ascend accelerators and X86/ARM CPUs, and .ms models for device-side inference on general-purpose CPUs and Kirin NPUs.

1. Device and Cloud one-stop inference deployment

   • Provide end-to-end workflow for model transformation optimization, deployment, and inference.

   • The unified IR realizes the device-cloud AI application integration.

2. Lightweight

   • Provides model compression, which could help to improve performance.

   • Provides the ultra-lightweight inference solution - MindSpore Lite Micro, to meet the deployment requirements in extreme environments such as smart watches and headphones.

3. High-performance

   • The built-in kernel computing library NNACL supports high-performance inference for dedicated chips such as CPU, NNRt, and Ascend, maximizing hardware computing power while minimizing inference latency and power consumption.

   • Use assembly instructions to improve performance of kernels.

4. Versatility

   • Supports deployment of multiple hardware such as server-side Ascend and CPU.

   • Supports HarmonyOS and Android mobile operating systems.

Further Understanding of MindSpore Lite

If you wish to further learn and use MindSpore Lite, please refer to the following content:

Supported models for cloud-side inference

Image/Video Generation Models	Vision-Language Models (VLM)	Large Language Models (LLM)	Audio Models (ASR/TTS)	Autonomous Driving / Embodied Intelligence	Information Retrieval / Embeddings / CNN / Others
Kandinsky-5.0-I2V-Lite-5s	Qwen3-VL-8B-Thinking	Qwen3.6-27B	WeNet	Mask2Former	Qwen3-Reranker-4B ✅
Kand0-T2V0-T2V-Lite-sft-10s	Qwen3-VL-8B-Instruct ✅	Qwen3.5-27B	FireRedASR-AED-L ✅	DinoV3	Qwen3-VL-Reranker-8B ✅
Kandinsky-5.0-T2I-Lite	Qwen3-VL-4B-Thinking ✅	Qwen3.5-9B	CosyVoice2-0.5B ✅	CenterPoint(2D)	Qwen3-VL-Reranker-2B ✅
Kandinsky-5.0-I2I-Lite	Qwen3-VL-4B-Instruct ✅	Qwen3.5-4B ✅	CosyVoice3-0.5B	CenterPoint(3D)	Qwen3-VL-Embedding-8B
Wan2.1-T2V-1.3B	Qwen3-VL-2B-Thinking ✅	Qwen3.5-2B ✅	Qwen3-ASR-0.6B	bevdet-r50 ✅	Qwen3-VL-Embedding-2B ✅
Wan2.1-T2V-14B	Qwen3-VL-2B-Instruct ✅	Qwen3.5-0.8B ✅	Qwen3-ASR-1.7B ✅	flashOCC	Qwen3-Reranker-0.6B ✅
Wan2.1-I2V-14B-480P	Qwen2.5-VL-3B-Instruct ✅	Qwen3-30B-A3B	Qwen3-TTS-12Hz-1.7B-Base	TinyVLA	jina-reranker-v3 ✅
Wan2.2-TI2V-5B	Qwen2-VL-2B-Instruct	Qwen3-8B ✅	Qwen3-TTS-12Hz-1.7B-CustomVoice ✅	GR00TN1.7	Qwen3-Embedding-0.6B ✅
Wan2.2-T2V-A14B	Qwen2-VL-2B	Qwen3-4B ✅	Qwen3-TTS-12Hz-1.7B-VoiceDesign	SpatialVLA	Qwen3-Embedding-4B ✅
Wan2.2-I2V-A14B	InternVL3_5-4B-Flash	Qwen3-1.7B ✅	Qwen3-TTS-12Hz-0.6B-Base	SmolVLA	yolov10x ✅
Wan2.2-Animate-14B	InternVL3_5-2B-Flash	Qwen3-0.6B ✅	Qwen3-TTS-12Hz-0.6B-CustomVoice	MiniVLA	vit-base-patch16-224 ✅
Qwen-Image-Edit	InternVL3_5-1B-Flash	Qwen2.5-7B-Instruct ✅		openVLA	bert-base-chinese ✅
Qwen-Image	InternVL3-2B	Qwen2.5-3B-Instruct ✅		openpi pi 0.5 ✅	Qwen2.5-Math-1.5B ✅
FLUX.1-dev	InternVL3-1B	Qwen2.5-0.5B ✅			Qwen2.5-Math-1.5B-Instruct ✅
stable-diffusion-v1-5	llava-v1.6	Qwen2-7B-Instruct ✅			Qwen3Guard-Gen-0.6B ✅
stable-diffusion-2-1	LLaVa	Qwen2-1.5B ✅			GLiNER-Large-v2.5 ✅
stable-diffusion-xl-base-1.0	BLIP	Qwen2-0.5B ✅			GLM-OCR ✅
	BLIP-2	Qwen1.5-moe-a2.7B			Grounding-DINO-Base ✅
	CLIP ✅				yolov8 ✅

API and documentation

1. API documentation:

   • C++ API documentation

   • Java API documentation

   • Python API documentation

   • HarmonyOS API Document

2. MindSpore Lite Official Website Document

Key characteristic capability

• Support Ascend hardware inference

• Supporting HarmonyOS

• Quantification after Training

• Lightweight Micro inference deployment

• Benchmark Debugging Tool

Additionally, MindSpore Lite cloud-side inference provides a standalone acceleration component, Lite Boost, to improve inference performance for cloud-side inference based on the PyTorch interface. For details, see the Lite Boost section.

Communication and Feedback

• Welcome to AtomGit Issues: submit questions, reports, and suggestions;

• Welcome to Community Forum: engage in technical and problem-solving exchanges;

• Welcome to Sig: to manage and improve workflow, participate in discussions and exchanges;

Surrounding communities

• MindSpore

• MindOne

• Mindyolo

• OpenHarmony