gemma-3-4b-it-q4_0-mnn: SIGSEGV in libMNN on Snapdragon 8 Gen 2 (MnnLlmChat 0.8.3 + master)

[dominostars]

Summary

taobao-mnn/gemma-3-4b-it-q4_0-mnn consistently SIGSEGVs in libMNN.so on a Snapdragon 8 Gen 2 / Android 13 device, reproduced via two completely independent harnesses:

1. Official MNN Chat Android app 0.8.3 (com.alibaba.mnnllm.android, Google Play release).
2. Custom llm_demo built from current MNN master (commit 86969a6a, 2026-05-22 — includes commit 9d76f8a "fix Gemma3 dual-RoPE export").

Smaller and non-gemma-3 models on the same device run fine — Qwen2.5-1.5B-Instruct-MNN runs at 513 ms median over a 500-sentence ja→en benchmark with 0% catastrophic output, ~1.45× faster than llama.cpp. Issue #3722 reported the same model crashing on the same hardware class; it was closed (stateReason: COMPLETED) but the failure is still present.

Environment

• Device: AYN Thor — Snapdragon 8 Gen 2 (Adreno 740), 16 GB RAM, Android 13.
• Model: taobao-mnn/gemma-3-4b-it-q4_0-mnn (default HF source from the Model Market; also downloaded directly).
• MNN: reproduced on tag 3.3.0 and master at 86969a6a (2026-05-22).
• NDK: 28.2.13676358; arm64-v8a; CMAKE_BUILD_TYPE=Release.

Reproduction 1 — MnnLlmChat 0.8.3

Open app → load gemma-3-4b-it-q4_0-mnn (either via Model Market download or /data/local/tmp/mnn_models/) → open chat → send any prompt → ~3 s of compute → native crash, chat activity dismissed.

F libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xb4000070a68dcfc0
  in tid 21209 (DefaultDispatch), pid 20900 (com.alibaba.mnnllm.android)
backtrace:
  #00 pc 0x383a40  libMNN.so  (BuildId: 32ec80c96d475d603f39b375f3f3b1196f162349)
  #01 pc 0x368974  libMNN.so

Reproduction 2 — llm_demo from master, MnnLlmChat-style flags

cmake -G Ninja -S MNN -B build \
  -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake \
  -DANDROID_ABI=arm64-v8a -DCMAKE_BUILD_TYPE=Release \
  -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON -DMNN_ARM82=ON -DMNN_OPENCL=ON \
  -DLLM_SUPPORT_VISION=true -DMNN_BUILD_OPENCV=true -DMNN_IMGCODECS=true \
  -DMNN_CPU_WEIGHT_DEQUANT_GEMM=true -DMNN_SUPPORT_TRANSFORMER_FUSE=true
cmake --build build --target llm_demo
adb push build/llm_demo build/libMNN.so /data/local/tmp/mnn/
adb shell "cd /data/local/tmp/mnn && LD_LIBRARY_PATH=. ./llm_demo gemma-3-4b-ref/config.json prompt.txt"
# → "config path is gemma-3-4b-ref/config.json"
# → Segmentation fault

Symbolized backtrace from an unstripped build:

#00  E1LoopH8                                              source/backend/cpu/arm/arm64/MNNPackedMatMulRemain.S:476
#01  StrassenMatrixComputor::_generateTrivalMatMul lambda  source/backend/cpu/compute/StrassenMatmulComputor.cpp:131

So the crash is in MNNPackedMatMulRemain.S (E1LoopH8), reached via StrassenMatrixComputor::_generateTrivalMatMul during the load-time geometry computation. The MnnLlmChat app defers this geometry compute until the first inference call, which is why the app shows ~3 s delay before crashing instead of crashing at load; llm_demo triggers it eagerly.

Ruled out (still crashes)

• MNN versions: 3.3.0 and 3.5.0 / master.
• Backends: backend_type: cpu and opencl — the crashing geometry compute runs on CPU regardless of the inference backend.
• Build flags: MNN_CPU_WEIGHT_DEQUANT_GEMM ON and OFF; with/without LLM_SUPPORT_VISION/MNN_BUILD_OPENCV/MNN_IMGCODECS and the -Wl,-z,max-page-size=16384 linker flag.
• Configs: precision: low / normal / high; use_template: true (default) and false; taobao's published config.json and a hand-written greedy variant.
• Model file: taobao's published llm.mnn/llm.mnn.weight and a fresh re-conversion via current llmexport (which already includes the 9d76f8a dual-rope export fix) — both crash.
• Conversion variants tried: int4 tied embed (default), 16-bit tied embed (--lm_quant_bit 16), --seperate_embed (bf16 embedding as a separate file).

Qwen2.5-1.5B-Instruct-MNN on the same device, same build, same llm_demo works perfectly (cleanly benchmarked at 513 ms median over 500 sentences). So this is not the harness, the device, or MNN-LLM in general — gemma-3-4b's specific shapes trigger a kernel bug.

Likely culprit

The MNNPackedMatMulRemain.S E1LoopH8 kernel — an ARM64 remainder / tile-boundary edge case triggered by gemma-3-4b's particular dimensions (vocab 262144, hidden 2560, head_dim 256, 4 KV heads via GQA). Possibly the same root cause as #3722; its COMPLETED status doesn't match the current behavior, since the model still crashes for users on this hardware (both #3722 and #4396 — also a Qualcomm flagship — reported similar). Happy to provide the full tombstone, run additional targeted tests, or test against a candidate patch.

[lzbiao8]

安卓手机上跑，回复都是乱七八糟的回复，有时候陷入死循环

[wangzhaode]

fixed in #4742