SYCL: segfault in ggml_sycl_add on Intel iGPU — regression between v1.8.2 and v1.8.3

[fjgalesloot]

SYCL: segfault in ggml_sycl_add on Intel iGPU — regression between v1.8.2 and v1.8.3

Summary

whisper-server built with the SYCL backend segfaults (SIGSEGV, exit 139) on the first inference request when running on an Intel Arrow Lake-U integrated GPU via Level Zero.

The crash was bisected to a single release: v1.8.2 works, v1.8.3 fails. Every release from v1.8.3 through v1.9.3 fails identically.

The fault is a host-side general protection fault inside libc, not a GPU hang — there are no i915 errors, engine resets, or EU page faults in dmesg.

Version

Bisected across official tags, all built identically:

whisper.cpp	Result
v1.7.4	works
v1.7.6	works
v1.8.0	works
v1.8.2	works
v1.8.3	crash
v1.8.4	crash
v1.8.7	crash
v1.9.2	crash
v1.9.3	crash

Operating systems

Linux (Ubuntu 24.04 container on Ubuntu host, kernel 6.8.0-111), Docker.

Which whisper.cpp modules do you know to be affected?

whisper-server (ggml-sycl backend)

Hardware

• Intel Arrow Lake-U integrated GPU, exposed to the container as an SR-IOV virtual function
• Reported by whisper.cpp as: [level_zero:gpu:0] Intel Graphics 12.70, 64 compute units, max work group 1024, max sub-group 32, 11296M global memory
• Level Zero driver version 1.6.33578+15
• Host kernel 6.8.0-111, i915 driver
• Device passed through: /dev/dri/renderD128

Model

ggml-large-v3-turbo.bin

Build

Built inside intel/oneapi:2025.3.1-0-devel-ubuntu24.04:

cmake -B build -DGGML_SYCL=ON -DGGML_NATIVE=OFF \
      -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx \
      -DGGML_SYCL_F16=ON
cmake --build build -j4 --config Release

Command line

build/bin/whisper-server -m /models/ggml-large-v3-turbo.bin --host 0.0.0.0 --port 8910

With SYCL_DEVICE_ALLOWLIST=BackendName:level_zero set. Decoding options (language, beam size, VAD) make no difference.

The server starts, enumerates the GPU, loads the model onto SYCL0 and allocates all compute buffers without error. It crashes on the first /inference request.

Crash location

With GGML_SYCL_DEBUG=1, the last three ops are:

[SYCL][OP] call ggml_sycl_im2col: dst='node_0':type=f16;ne=[384, 3000, 1, 1] ...
[SYCL][OP] call ggml_sycl_im2col done
[SYCL][OP] call ggml_sycl_mul_mat: dst='node_3':type=f32;ne=[3000, 1280, 1, 1] ...
[SYCL][OP] call ggml_sycl_mul_mat done
[SYCL][OP] call ggml_sycl_add: dst='node_5':type=f32;ne=[3000, 1280, 1, 1];nb=[4, 12000, 15360000, 15360000]
                               src0=' (reshaped)':type=f32;ne=[3000, 1280, 1, 1];nb=[4, 12000, 15360000, 15360000]
                               src1='leaf_3':type=f32;ne=[1, 1280, 1, 1];nb=[4, 4, 5120, 5120]
<segfault — no "done">

So it dies inside ggml_sycl_add, on the conv1 bias add: a broadcast of [1, 1280, 1, 1] against [3000, 1280, 1, 1]. This is the third op of the encoder — before any attention op runs.

Corresponding host kernel log:

traps: whisper-server[448759] general protection fault ip:7f281b1ecfeb sp:7f27c11c6578
       error:0 in libc.so.6[7f281b08a000+188000]

No i915 GPU hang / reset / page-fault messages accompany it.

Ruled out

Tested on some of the failing builds; none of these prevent the crash:

• GGML_SYCL_ENABLE_FLASH_ATTN=0 (confirmed applied — all FLASH_ATTN_EXT support checks return false, and the encode compute buffer grows from 55.35 MB to 78.57 MB)
• GGML_SYCL_FA_ONEDNN=0
• GGML_SYCL_ENABLE_FUSION=0
• GGML_SYCL_USE_ASYNC_MEM_OP=0
• GGML_SYCL_DEV2DEV_MEMCPY=2
• GGML_SYCL_DISABLE_OPT=1
• Rebuilding with -DGGML_SYCL_F16=OFF

Possible cause

#3492 is the only change in v1.8.3 that touches GPU device selection, and the failing builds log the device type it added:

whisper_backend_init_gpu: device 0: SYCL0 (type: 1)
whisper_backend_init_gpu: found GPU device 0: SYCL0 (type: 1, cnt: 0)

I have not verified it is the cause — it is a correlation, not a conclusion.

Notes

Happy to run further diagnostics, test patches, or provide a full GGML_SYCL_DEBUG=1 log or core dump backtrace.