Whisper.cpp Vulkan on Adreno 840 (Snapdragon 8 Elite Gen 5) in Termux — 4.3× faster than CPU, with a shaderc coopmat2 build fix

[xidoc]

Whisper.cpp Vulkan on Adreno 840 (Snapdragon 8 Elite Gen 5) in Termux — 4.3× faster than CPU, with a shaderc coopmat2 build fix

TL;DR

I got whisper.cpp running on the GPU (Adreno 840) of an Honor Magic 8 Pro (Snapdragon 8 Elite Gen 5) inside Termux, no root, using the Turnip/Mesa Vulkan driver.

Result: the GPU build transcribes ~4.3× faster than the (already very fast) CPU build.

Backend	120 s of audio, small model	real time
GPU — Vulkan (Adreno 840, Turnip)	whisper-cli -m small	0m27.8s
CPU — 8 Oryon V3 cores	whisper-cli -m small -t 6 -p 2	2m00.8s

To get there I had to work around a shaderc build failure on the cooperative-matrix-2 (cm2) shaders. That failure — and my workaround — is the main reason I'm posting: I suspect the root cause is a false-positive feature-detection in the Vulkan CMake, and a maintainer may want to fix it properly.

I'm not sure whether the "correct" home for this is whisper.cpp, ggml, or the shaderc packaging in Termux, so I'm posting here and happy to move it.

---

1. Hardware / software

• Phone: Honor Magic 8 Pro
• SoC: Qualcomm Snapdragon 8 Elite Gen 5 (SM8850-AC), 3 nm
• CPU: Octa-core Oryon V3 — 2× Phoenix L @ 4.6 GHz + 6× Phoenix M @ 3.62 GHz
• GPU: Adreno 840
• RAM: 12 GB LPDDR5X, UFS 4.1 storage
• OS: Android 16 / MagicOS 10
• Termux: from F-Droid (NOT the Play Store build)
• Vulkan loader: libvulkan.so 1.4.356
• Driver used for compute: Turnip / Mesa (freedreno) — the stock Qualcomm Vulkan driver does not work for these shaders (details below)
• clang: 21.x (Termux NDK toolchain)
• whisper.cpp commit: 080bbbe8
• shaderc / glslc: current Termux package (2025-era), which is where the build breaks

GPU as reported by the working build:

ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Adreno (TM) 840 (turnip Mesa driver) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 32768 | int dot: 0 | matrix cores: none

Note matrix cores: none — the Adreno has no usable cooperative-matrix hardware exposed through Vulkan. This is central: the cm2 shaders should arguably never be generated for this device, but the build tries to compile them anyway and crashes.

---

2. The use case (context for why I was building this)

End goal: a one-command pipeline on the phone that:

1. downloads a YouTube video or playlist (yt-dlp),
2. transcribes the English audio locally (whisper.cpp),
3. auto-translates the subtitles to French (deep-translator),
4. muxes English + French soft-subs back into the MP4 (ffmpeg).

Everything 100% on-device. Whisper transcription was by far the slowest stage, which is why GPU offload mattered.

---

3. The build problem: shaderc internal error: Invalid capability operand: 5447

A straight -DGGML_VULKAN=ON build fails while generating SPIR-V shaders. The failing shader is always a cooperative-matrix-2 one, e.g. flash_attn_cm2 / matmul_*_cm2:

cannot compile flash_attn_f32_f16_f16acc_cm2 (exit code 1)

/data/data/com.termux/files/usr/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 \
  .../vulkan-shaders/flash_attn_cm2.comp -o .../flash_attn_f32_f16_f16acc_cm2.spv -O \
  -DACC_TYPE=float16_t ... -DFLOAT16=1 ...

shaderc: internal error: compilation succeeded but failed to optimize: Invalid capability operand: 5447

vulkan-shaders-gen: one or more shaders failed to compile

5447 is the SPIR-V capability operand for CooperativeMatrixKHR. The glslc/shaderc build shipped in Termux compiles the shader but then fails during the optimizer pass on that capability.

This looks like the same class of problem as ggml-org/llama.cpp#15344 ("shaderc v2025.2 causes vulkan compilation failure"), where a newer shaderc + a false-positive GL_EXT_bfloat16 / coopmat feature test leads to generating shaders that then can't be optimized. In that issue the nixpkgs folks worked around it by patching out the bfloat16 shader feature test.

On my machine the CMake feature test test_shader_extension_support(... "GL_NV_cooperative_matrix2" ...) returns success (a false positive) — so GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets defined, the generator emits cm2 shaders, and then glslc dies optimizing them. The hardware reports matrix cores: none, so the test arguably should fail here.

---

4. What did NOT work (for the record, so others don't waste time)

• Passing -DGGML_VULKAN_COOPMAT2_GLSLC_SUPPORT=OFF on the CMake command line → ignored; the in-tree test_shader_extension_support() re-enables it.
• Adding set(GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT OFF CACHE BOOL "" FORCE) after the feature tests → too late; the test has already emitted the compile definition, and vulkan-shaders-gen (a separately-built helper) still generates cm2 shaders.
• Disabling only in the generator, or only in the C++, → desync: if you stop generating cm2 but the C++ still references matmul_*_cm2_len/_data, you get
  error: 'matmul_bf16_cm2_len' was not declared in this scope.
• The stock Qualcomm Vulkan driver segfaults on these compute pipelines regardless (Compute pipeline creation failed for adreno_mul_mat_vec_q4_k_q8_1_f32), which is why Turnip is required.
• Also unrelated but worth noting: Android's phantom process killer and background-app killing will SIGKILL (signal 9) a long compile. Keep Termux in the foreground, acquire the wake-lock, and build with -j2 to keep RAM pressure down.

---

5. What DID work — the workaround

The reliable fix is to make sure COOPMAT2 is disabled consistently on both sides (shader generator and the C++ consumer), so nothing references shaders that were never generated. I did this by neutralizing the two coopmat2 feature tests in the Vulkan CMakeLists.txt, plus hard-disabling the #if blocks in the generator as a belt-and-braces measure.

5a. ggml/src/ggml-vulkan/CMakeLists.txt

Replace the two coopmat2 feature-test calls:

    test_shader_extension_support(
        "GL_NV_cooperative_matrix2"
        "${CMAKE_CURRENT_SOURCE_DIR}/vulkan-shaders/feature-tests/coopmat2.comp"
        "GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT"
    )

    test_shader_extension_support(
        "GL_NV_cooperative_matrix_decode_vector"
        "${CMAKE_CURRENT_SOURCE_DIR}/vulkan-shaders/feature-tests/coopmat2_decode_vector.comp"
        "GGML_VULKAN_COOPMAT2_DECODE_VECTOR_GLSLC_SUPPORT"
    )

with hard-off settings (so the compile definition is never emitted):

    # WORKAROUND: coopmat2 disabled — Adreno 840 has no matrix cores and
    # shaderc (Termux) crashes optimizing cm2 shaders (Invalid capability operand: 5447)
    set(GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT OFF)
    set(GGML_VULKAN_COOPMAT2_DECODE_VECTOR_GLSLC_SUPPORT OFF)

5b. ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp

Force every cm2 generation block off, so the generator can't emit them even if the macro leaks in from somewhere:

sed -i 's/#if defined(GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT)/#if defined(GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT) \&\& 0/g' \
  ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp

5c. Clean rebuild (important)

vulkan-shaders-gen is compiled early and cached; a partial rebuild will reuse the old generator. Always wipe build/:

cd ~/whisper-gpu
rm -rf build
export ASAN_OPTIONS=detect_leaks=0    # avoids an Android 11+ tagged-pointer abort
cmake -B build \
  -DGGML_VULKAN=ON \
  -DGGML_NATIVE=OFF \
  -DGGML_CPU_ARM_ARCH="armv8.2-a+dotprod+i8mm" \
  -DWHISPER_BUILD_TESTS=OFF \
  -DWHISPER_BUILD_EXAMPLES=ON
cmake --build build -j2 --config Release

With cm2 gone, the build completes. cm1 (cooperative-matrix v1) and the scalar/fp16 paths are still generated and used.

Note: -DGGML_NATIVE=OFF + explicit -DGGML_CPU_ARM_ARCH="armv8.2-a+dotprod+i8mm" is a separate Termux fix — the CPU backend's native detection enables sve/sme, and clang 21 segfaults (exit 139) compiling ggml_gemm_q8_0_4x8_q8_0 in repack.cpp. Pinning to armv8.2-a+dotprod+i8mm avoids that.

---

6. Full reproducible setup (from scratch)

6a. Toolchain + Turnip driver

pkg update && pkg upgrade -y
pkg install -y git cmake clang make ninja
pkg install -y x11-repo tur-repo
pkg install -y vulkan-loader-generic vulkan-headers shaderc glslang \
               spirv-headers spirv-tools vulkan-tools \
               mesa-vulkan-icd-freedreno-dri3

6b. Clone + patch + build (see section 5)

cd ~
git clone https://github.com/ggml-org/whisper.cpp whisper-gpu
cd whisper-gpu
# apply 5a + 5b, then run the clean rebuild from 5c
./models/download-ggml-model.sh small

6c. Run on GPU

The stock Qualcomm ICD must be overridden with Turnip at runtime:

export VK_ICD_FILENAMES=$PREFIX/share/vulkan/icd.d/freedreno_icd.aarch64.json
export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH

./build/bin/whisper-cli -m ./models/ggml-small.bin -f audio16k.wav -pp -osrt -of out

Success line to look for:

ggml_vulkan: 0 = Adreno (TM) 840 (turnip Mesa driver) | uma: 1 | fp16: 1 ...
whisper_backend_init_gpu: using Vulkan0 backend

---

7. Benchmark detail

Same 120 s WAV (16 kHz mono), ggml-small.bin, warm cache:

=== GPU (Vulkan Adreno 840, Turnip) ===
real    0m27.758s
user    0m8.354s
sys     0m2.449s

=== CPU (6 threads -p 2, Oryon V3) ===
real    2m0.821s
user    7m22.095s
sys     4m24.979s

~4.3× faster on GPU. For context, the CPU here is not weak — it's one of the fastest mobile CPUs available, and I'd already tuned it (-t 6 -p 2 beat -t 8, likely memory-bus contention). The GPU still wins comfortably.

Interesting because a lot of Adreno + llama.cpp reports show the opposite (GPU slower than CPU on mobile, due to shared-memory bandwidth). That seems to be workload-dependent: Whisper's encoder is compute-heavy matmul that maps well to the Adreno, whereas autoregressive LLM decoding is memory-bound and suffers. So "Adreno GPU is pointless for on-device inference" is not a safe generalization — for Whisper on this SoC it's a big win.

---

8. Questions for maintainers

1. Should test_shader_extension_support() for coopmat2 be treated as a hard capability check against the device (which reports matrix cores: none) rather than a glslc-only check? The glslc test passing while the device lacks the capability is what leads to generating unusable shaders.
2. Is the shaderc-optimizer crash on capability 5447 known/tracked upstream (mirrors llama.cpp#15344 for bf16/coopmat)? A guard that skips optimization or the shader when the target lacks the capability would remove the need for downstream patches.
3. Would a config option like -DGGML_VULKAN_DISABLE_COOPMAT2=ON that reliably propagates to both vulkan-shaders-gen and ggml-vulkan.cpp be acceptable? Right now disabling it consistently requires editing two files.

Happy to test patches on real Adreno 840 hardware — glad to help validate a proper fix.

---

Everything above was done on a stock, non-rooted Honor Magic 8 Pro. Thanks for whisper.cpp — the Vulkan backend genuinely turns a flagship phone into a fast local transcription box.

---

Disclosure: I'm not a Vulkan/GPU expert. I worked through this debugging session with the help of an AI assistant (Claude), but every command was run on my own physical Honor Magic 8 Pro, and all logs, error messages, and benchmark numbers above are real output captured from that device — nothing here is synthetic or untested. I'm posting the full trace so the result is verifiable and reproducible, and I'm happy to re-run anything or test proposed patches on real Adreno 840 hardware.

Appendix — full pipeline script (ytdl)

#!/data/data/com.termux/files/usr/bin/bash
#
# ytdl — on-device YouTube → transcribe (EN) → translate (FR) → mux subtitles
# ---------------------------------------------------------------------------
# Runs entirely on an Android phone via Termux, using whisper.cpp on the GPU
# (Adreno via Turnip/Mesa Vulkan). Tested on Honor Magic 8 Pro / Snapdragon
# 8 Elite Gen 5 / Adreno 840. See discussion:
#   https://github.com/ggml-org/whisper.cpp/discussions/3943
#
# PIPELINE PER VIDEO:
#   1. yt-dlp downloads best video+audio, merges to mp4 (SponsorBlock trimmed)
#   2. ffmpeg extracts 16 kHz mono WAV (what whisper wants)
#   3. whisper.cpp (GPU build) transcribes -> <name>.en.srt
#   4. deep-translator (Google) translates each subtitle line -> <name>.fr.srt
#   5. ffmpeg muxes EN + FR soft-subs back into <name>_ST.mp4
#
# USAGE:
#   ./ytdl "https://youtu.be/XXXX"            # default model: small
#   ./ytdl "https://youtu.be/XXXX" medium     # higher quality
#   ./ytdl "https://youtube.com/playlist?..." # playlists work too
#
# PREREQUISITES (one-time), see the discussion linked above for the full
# build instructions of the GPU whisper.cpp in ~/whisper-gpu:
#   pkg install python ffmpeg
#   pip install yt-dlp deep-translator
#   ./models/download-ggml-model.sh small   (and/or medium)
# ---------------------------------------------------------------------------

set -e

# --- GPU (Vulkan / Turnip) runtime setup ---------------------------------
# The stock Qualcomm Vulkan driver crashes on these compute pipelines, so we
# force the open-source Turnip (Mesa) driver at runtime.
export VK_ICD_FILENAMES=$PREFIX/share/vulkan/icd.d/freedreno_icd.aarch64.json
export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH

URL="$1"
SIZE="${2:-small}"                                  # model size: tiny/base/small/medium/large-v3
WH=~/whisper-gpu/build/bin/whisper-cli              # GPU-enabled whisper.cpp build
MODEL=~/whisper.cpp/models/ggml-${SIZE}.bin
OUT=~/storage/shared/YTDL                           # lands in Internal storage/YTDL

if [ -z "$URL" ]; then
  echo "Usage: ./ytdl \"<youtube url>\" [model_size]"
  exit 1
fi

mkdir -p "$OUT"; cd "$OUT"

fmt(){ printf '%dm%02ds' $(($1/60)) $(($1%60)); }   # pretty-print seconds
T0=$(date +%s)

echo ">>> Model: $SIZE | GPU Adreno (Vulkan/Turnip)"
echo ">>> Downloading..."
t=$(date +%s)
yt-dlp -f "bv*+ba/b" --merge-output-format mp4 \
  --sponsorblock-remove sponsor -N 8 \
  -o "%(title)s.%(ext)s" "$URL"
echo "    [download: $(fmt $(($(date +%s)-t)))]"

for f in *.mp4; do
  [ -f "$f" ] || continue
  base="${f%.*}"
  case "$base" in *_ST) continue;; esac            # skip already-processed files
  [ -f "${base}_ST.mp4" ] && continue

  echo ">>> [$base]"

  echo "    extracting 16 kHz audio..."
  t=$(date +%s)
  ffmpeg -y -i "$f" -ar 16000 -ac 1 -c:a pcm_s16le "$base.wav" -loglevel error
  echo "    [audio: $(fmt $(($(date +%s)-t)))]"

  echo "    transcribing on GPU..."
  t=$(date +%s)
  "$WH" -m "$MODEL" -f "$base.wav" -l auto -pp -osrt -of "$base.en"
  echo "    [transcription: $(fmt $(($(date +%s)-t)))]"
  rm -f "$base.wav"

  echo "    translating EN -> FR..."
  t=$(date +%s)
  python - "$base" <<'PY'
import sys
from deep_translator import GoogleTranslator
b = sys.argv[1]
try:
    lines = open(b + ".en.srt", encoding="utf-8").read().split("\n")
    tr = GoogleTranslator(source="en", target="fr")
    total = sum(1 for l in lines if l.strip() and not l.strip().isdigit() and "-->" not in l)
    out = []; n = 0
    for l in lines:
        if l.strip() and not l.strip().isdigit() and "-->" not in l:
            n += 1
            print(f"\r    translating: {n}/{total}", end="", flush=True)
            try:
                out.append(tr.translate(l))
            except Exception:
                out.append(l)          # keep English line if a request fails
        else:
            out.append(l)              # keep indices, timestamps, blank lines
    print()
    open(b + ".fr.srt", "w", encoding="utf-8").write("\n".join(out))
except FileNotFoundError:
    pass
PY
  echo "    [translation: $(fmt $(($(date +%s)-t)))]"

  echo "    muxing EN + FR soft-subs..."
  t=$(date +%s)
  ffmpeg -y -i "$f" -i "$base.en.srt" -i "$base.fr.srt" \
    -map 0 -map 1 -map 2 -c copy -c:s mov_text \
    -metadata:s:s:0 language=eng -metadata:s:s:0 title="English" \
    -metadata:s:s:1 language=fre -metadata:s:s:1 title="Francais" \
    "${base}_ST.mp4" -loglevel error && rm -f "$f"
  echo "    [mux: $(fmt $(($(date +%s)-t)))]"

  echo "    >>> DONE"
done

echo "=== Finished in $(fmt $(($(date +%s)-T0))). Files in $OUT ==="

[0cc4m]

The glslc test passing while the device lacks the capability is what leads to generating unusable shaders.

The test just checks whether your glslc compiler supports it, not your device. That is a runtime check. You probably just need to update your shader compiler.

[xidoc]

Thanks for the clarification! That makes perfect sense. You are absolutely right that the CMake test only checks the compiler capabilities, and the crash is due to a bug in the specific glslc version during the optimization pass.

​The limitation here is that in the Termux ecosystem on Android, we are entirely dependent on the pre-built packages currently available in their repositories. The current shaderc package provided for Termux is older and contains this optimization bug. Updating the compiler isn't as straightforward as it is on a standard Linux desktop unless a user compiles the entire toolchain from scratch.

​Given that mobile environments might often be stuck with slightly outdated or buggy compiler packages, do you think there's room for a CMake option (e.g., -DGGML_VULKAN_DISABLE_COOPMAT2=ON) to manually bypass the generation of these specific shaders? For now, my local workaround does the job, but an official flag could save a lot of headaches for others trying to leverage their phone's GPU for local inference.