sync : ggml - #3962

Merged

danbev merged 102 commits into

sync-ggml-26-07-30

Jul 31, 2026

Commits on Jul 30, 2026

hexagon: tiling, tracing and optimizations for unary ops (llama/25474)

authored and
ggerganov
committed
872f31b
ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (llama/24776)

committed
5635f4b
opencl: cluster-parallel decode FA for Adreno (llama/25473)

wanghqc
authored and
ggerganov
committed
f865ccc
ggml-et: Initial ET backend (llama/24179)

authored and
ggerganov
committed
9d65d9f
hexagon: improve ARGSORT performance for small tensors (llama/25512)

max-krasnyansky
authored and
ggerganov
committed
e1993f4
opencl: add int8 dp4 dense and MoE prefill optimization for Adreno GPUs (llama/25537)

authored and
ggerganov
committed
ea80853
Vulkan: route large matmuls to medium tile on Adreno (llama/24877)

Raman-Raje
authored and
ggerganov
committed
4bc66b5
ggml : add GGML_OP_LIGHTNING_INDEXER that implements DeepSeek V3.2/V4 lightning indexer (llama/24231)

authored and
ggerganov
committed
19c96ad
cuda: Don't crash when querying memory on device with no free memory. (llama/25157)

cphlipot
authored and
ggerganov
committed
f00dc30
gguf : reject empty metadata keys (llama/24917)

rohitmahesh1
authored and
ggerganov
committed
dbe941f
sycl: add Q2_K to DMMV reorder path (llama/25064)

malsbat
authored and
ggerganov
committed
56e0229
sycl: add fused top-k MoE (llama/25217)

authored and
ggerganov
committed
6b3f122
gguf : add tensor shape accessor (llama/24405)

QuintinShaw
authored and
ggerganov
committed
56bccf6
vulkan: Use native e2m1 and e4m3 conversions for mxfp4/nvfp4 (llama/25338)

jeffbolznv
authored and
ggerganov
committed
9a18c7c
CUDA: refactor MMQ kernel configuration (llama/24127)

JohannesGaessler
authored and
ggerganov
committed
dac3141
metal : add Q2_0 support (llama/25419)

khosravipasha
authored and
ggerganov
committed
f5f0b64
sycl: set fattn_vec_nthreads to 256 for Battlemage (llama/25205)

Titaniumtown
authored and
ggerganov
committed
e4567a6
kleidiai : add SME2 f32 kernel (llama/24414)

chaxu01
authored and
ggerganov
committed
7f75229
ggml: uniformize im2col dst_type for all conv ops (llama/23660)

Juste-Leo2
and
ggerganov
committed
c766b6b
ggml : add a set of functions for checking contiguity of inner tensor dimensions (llama/25650)

authored and
ggerganov
committed
387aa89
vulkan/cpu: Support f16 as SET_ROWS src. (llama/25432)

jeffbolznv
authored and
ggerganov
committed
d4c391b
opencl: fix a dp4a bug for devices where cl_khr_integer_dot_product is unavailable (llama/25639)

authored and
ggerganov
committed
0ffd499
hexagon: fix hmx-queue signal enum-narrowing problem (llama/25677)

chyan8
authored and
ggerganov
committed
d3cdee1
opencl: avoid the vec path in GEMV for unaligned row stride (llama/25671)

wanghqc
authored and
ggerganov
committed
d44b96c
opencl: handle OOB write in noshuffle GEMV kernels (odd ne01) (llama/25640)

wanghqc
authored and
ggerganov
committed
ff6847f
opencl: do not use `clCreateBufferWithProperties` when targeting CL 2.x (llama/25673)

wanghqc
authored and
ggerganov
committed
a46de2e
Flash Attention with XMX engine via oneDNN (llama/25222)

authored and
ggerganov
committed
4ff381d
sycl: Increase minimum buffer size for USM system allocations (llama/25525)

ifdu
authored and
ggerganov
committed
af1fe65
sycl : implement xielu op (llama/25550)

atsmith3
authored and
ggerganov
committed
15b52ca
sycl : support kernel type fp16 for conv2d_dw (llama/25653)

arthw
authored and
ggerganov
committed
b446531
sycl : fix get_rows Q2_K, Q4_K, Q5_K (llama/25656)

arthw
authored and
ggerganov
committed
af80afa
ggml: add f16 out_prod support for CPU and out_prod op for Vulkan (llama/23997)

Lamothe
authored and
ggerganov
committed
aadab83
metal: fuse snake activation (mul, sin, sqr, mul, add) (llama/25459)

ServeurpersoCom
authored and
ggerganov
committed
e376ba3
cuda : relax tensor contiguity requirements for quantized concat (llama/25678)

authored and
ggerganov
committed
8096ba6
CUDA: tighter MMQ src1 buffer size for native fp4 (llama/25613)

leonardHONG
authored and
ggerganov
committed
f9d97c3
opencl: fix two issues on flash attention for Adreno a7x (llama/25697)

wanghqc
authored and
ggerganov
committed
c4a15ec
cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) (llama/25545)

authored and
ggerganov
committed
7ea23eb
opencl: exclude some moe kernels on Adreno a7x (llama/25698)

wanghqc
authored and
ggerganov
committed
b8c8175
cuda: extract Q1_0 elements via __byte_perm (llama/25628)

dfriehs
authored and
ggerganov
committed
797cccb
opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU (llama/25745)

authored and
ggerganov
committed
ea8708a
CUDA: dedup MoE gate/up activation quantization (llama/25441)

praneshgo
authored and
ggerganov
committed
cf26eac
ggml-cuda : restore prop.integrated on HIP builds (llama/24233)

liminfei-amd
authored and
ggerganov
committed
73c71d1
Enable CUDA graphs on volta+turing (llama/25749)

heislera763
authored and
ggerganov
committed
aa3743a
CUDA: Support CUDA Virtual Devices (llama/25228)

anavp-nvidia
authored and
ggerganov
committed
b4c0492
vulkan: when using transfer queue for async copies, sync on event_wait to avoid race (llama/25229)

0cc4m
authored and
ggerganov
committed
913750f
kleidiai: Add SME vs SME2 distinction in kernel dispatch (llama/25478)

matcraje
authored and
ggerganov
committed
b90a4f4
hexagon: L2 cache handling rework (dirty bit tracking with lazy flushing) and more MUL_MAT updates (llama/25762)

max-krasnyansky
authored and
ggerganov
committed
1a4b5d1
DeepseekV4: Add fused hyper-connection ops (llama/25585)

am17an
authored and
ggerganov
committed
070e572
docs: added a note about using OpenCl with Adreno 810 (llama/25786)

akleine
authored and
ggerganov
committed
5e5148f
opencl: loads quants as uint for q4_K and q5_K flat mv (optimization for Adreno A7x GPUs) (llama/25780)

authored and
ggerganov
committed
4dbbaef
opencl: add ABS op (llama/25115)

Gezahegne
authored and
ggerganov
committed
4db5bc4
sycl: fix row calculation when K_QUANTS_PER_ITERATION is 1 (llama/25690)

malsbat
authored and
ggerganov
committed
ecbac66
vulkan: Support Q2_0 (llama/25430)

jeffbolznv
authored and
ggerganov
committed
ca31169
ggml-blas: default hadamard mul_mat to cpu routine (llama/25710)

taronaeo
authored and
ggerganov
committed
6c8bac6
ggml : bump version to 0.17.0 (ggml/1568)
ggerganov
committed
9958d50
opencl: transpose q4_K noshuffle scales for coalesced reads (llama/25805)

wanghqc
authored and
ggerganov
committed
7ff3d47
opencl: read/write MoE dp4a activation tiles to local memory as 128-bit (vectorized LD/ST perf opt) for Adreno GPUs (llama/25810)

wanghqc
authored and
ggerganov
committed
1e7873f
opencl: load and use `kernel_gemm_moe_q6_k_f32_ns` from bin kernel lib (llama/25797)

lhez
authored and
ggerganov
committed
a23301f
opencl: Support broadcast for Adreno MUL_MAT and honor `view_offs` for Adreno Q8_0 MUL_MAT for llama-server multi-stream (llama/25910)

authored and
ggerganov
committed
6963f93
hexagon: add CLAMP op (llama/25934)

tboinovski1
authored and
ggerganov
committed
5269754
CUDA: vectorize same-type get_rows with int4 copy (llama/25929)

pwilkin
authored and
ggerganov
committed
9747391
ggml-openvino: Add GGML_BACKEND_DL_IMPL invocation for OpenVINO backend (llama/25795)

seijikun
authored and
ggerganov
committed
2fd0abd
vulkan: Refactor vk_queue to use per-instance mutexes and unique handles (llama/23570)

authored and
ggerganov
committed
338e07d
kleidiai : warn once when a weight type has no KleidiAI kernel (llama/25701)

kkjjkamal123
authored and
ggerganov
committed
812ad39
cuda: add sqrt_softplus in topk-moe for dsv4 (llama/25896)

am17an
authored and
ggerganov
committed
7c9d38e
hexagon: check tensor type when reusing descriptors (llama/25968)

w1049
authored and
ggerganov
committed
9145f47
cuda: GET_ROWS quants (llama/25962)

ServeurpersoCom
authored and
ggerganov
committed
f74102e
webgpu : add CONV_2D_DW (depthwise conv2d) kernel (llama/25847)

m1el
authored and
ggerganov
committed
fcb2236
ggml: enable PowerPC backend variants on AIX (llama/25983)

shalinib-ibm
authored and
ggerganov
committed
4d546f4
hexagon: activation ops update (llama/25974)

authored and
ggerganov
committed
613a693
CUDA: Improve NVFP4 W4A4 activation quantization (llama/25730)

ORippler
authored and
ggerganov
committed
818f9e9
metal : add f16 type support to leaky relu (llama/25981)

iliailmer
authored and
ggerganov
committed
5716870
CUDA: fix external compilation of q1_0 MMQ (llama/25778)

JohannesGaessler
authored and
ggerganov
committed
49d48de
hexagon: fix Windows crash when op_poll is enabled (llama/26029)

adgup
authored and
ggerganov
committed
47a3239
hexagon: further improved pipeline of the core bits (L2, DMA, MM, FA) (llama/26049)

max-krasnyansky
authored and
ggerganov
committed
ddc63be
opencl: do not treat NULL-mask flash attention as causal (llama/25771)

kumaal
authored and
ggerganov
committed
ce73581
opencl: cache compiled cl_program binaries on disk (llama/26050)

wanghqc
authored and
ggerganov
committed
526d0ed
HIP: remove rocWMMA FlashAttention (llama/26046)

JohannesGaessler
authored and
ggerganov
committed
9dbbfa4
Update ggml/src/gguf.cpp : Defined virtual keyword for destructor of gguf_writer_base (llama/25867)

yymin1022
authored and
ggerganov
committed
6d4dc97
hexagon: partial im2col support (llama/26007)

tboinovski1
authored and
ggerganov
committed
a798a7e
opencl: fix fused RMS norm mul view offset (llama/26085)

happyyzy
authored and
ggerganov
committed
2e5b78c
ggml-cpu: Enable BF16 tiled gemm optimization on PowerPC (llama/26068)

shalinib-ibm
authored and
ggerganov
committed
98a532e
ggml : adjust logic for offloading ops to weight's backend (llama/25832)
ggerganov
committed
2fa1559
sycl(build): parallelize ocloc invocations (llama/25903)

Titaniumtown
authored and
ggerganov
committed
7ebe1c2
Disable -ffast-math on HIP (llama/25495)

Beinsezii
authored and
ggerganov
committed
0c1bd5d
ggml-metal: FWHT kernel for metal backend (llama/25924)

committed
00009c8
sycl: fix use-after-return of the SDPA scale in the oneDNN flash-attention path (llama/25880)

authored and
ggerganov
committed
e43559f
ggml-cuda: add chunked SSD matmul for Mamba-2 prefill acceleration (llama/22675)

BLSharda
authored and
ggerganov
committed
1e4d28c
vulkan: add iq4_nl support back to FA (llama/24585)

jeffbolznv
authored and
ggerganov
committed
cb8a0ad
ggml : set output of view src (llama/25729)

committed
98dc176
opencl: skip the Adreno KQ/KQV image kernels for multi-stream batches (llama/26189)

wanghqc
authored and
ggerganov
committed
3583bc3
ggml-webgpu: Fix some binding alias issues to support all archs, fix recurrent-state-rollback test (llama/25931)

reeselevine
authored and
ggerganov
committed
2709c63
add rdna3.5, and 3 to mmq configs so they can be tuned independently. (llama/26199)

Geramy
authored and
ggerganov
committed
3ab356f
RPC: add tensor_memset (llama/25912)

am17an
authored and
ggerganov
committed
58e109a
sycl: contiguous fast path + 32-bit index math for unary elementwise ops (llama/25946)

Titaniumtown
authored and
ggerganov
committed
e103c72
ggml-cuda : disable MMQ on devices with less than 48 KiB shared memory (llama/26141)

KakaruHayate
authored and
ggerganov
committed
a816f40
ggml : Fix issue with kleidiai ci and stringop overflow warning (llama/26277)

JonathanC-ARM
authored and
ggerganov
committed
c2b2ecb
metal: fix memory unwire if model is freed without any GPU operations (llama/26082)

authored and
ggerganov
committed
2b8f0f1
CUDA: add Q2_0 support (llama/25707)

khosravipasha
authored and
ggerganov
committed
6366ca0
ggml : bump version to 0.18.0 (ggml/1576)
ggerganov
committed
6f6960d
sync : ggml
ggerganov
committed
2f1f1a8

Commits on Jul 31, 2026

parakeet : update parakeet test models generation
danbev
committed
0aec3b2