perf(gqa): template the sliding window out of the no-window v5 prefill path (-494 ms TTFT)

[BoarQing]

Stacked on #622 (base is perf/gqa-flash-prefill-window), so the diff here is one commit against one file.

The problem

#622 added the sliding window as a runtime argument. Its per-row state — the kv_lo lower bound, the all-masked-row empty guard and the selects it drives — therefore stays live in every instantiation, whether or not a window is configured. gqa_flash_prefill_v5_kernel already runs at the VGPR ceiling, so in the instantiation the autotuner picks for gpt-oss that state spills:

M_TILES=2 BKV=32 D=64	before #622	with #622
scratch (bytes/lane)	120	180
VGPR spills	38	68

The cost lands on the full-attention path, which gained nothing from the window feature. -Rpass-analysis=kernel-resource-usage, gfx1151.

The change

bool HAS_WINDOW becomes a fourth template parameter, and the three window sites go under if constexpr: the start-tile clamp, the per-row kv_lo mask in the innermost loop, and the all-masked-row short-circuit. local_window_size stays a runtime argument, since the windowed instantiation still needs its value; only the branches become compile-time.

dispatchPrefillV5 selects on local_window_size > 0. Only D == 64 gets a windowed instantiation, because hip_gqa_flash_prefill_v3 declines a window at any other head dim, so a D=128 windowed variant would be dead code. That puts the instantiation count at 12 rather than 16.

No autotuner change is needed: the cache key already carries _w<window>, so each instantiation tunes itself.

Results

Every HAS_WINDOW=false instantiation is byte-identical to the pre-window kernel — the goal was parity, not improvement:

	pre-window	#622	this PR
M2 B32 D64 scratch B/lane	120	180	120
M2 B32 D64 VGPR spills	38	68	38
M1 B32 D64 VGPRs / occupancy	188 / 8	196 / 7	188 / 8

The HAS_WINDOW=true instantiations reproduce #622's numbers exactly, as expected: their branches become unconditional rather than removed.

Standalone prefill harness, interleaved arms, 4 rounds, only custom_kernels_gfx1151.dll differing (ms):

case	pre-window	#622	this PR
gpt_oss-sink sq512 past8192 (full attention, deep KV)	5.16	6.29	5.16
gpt_oss-sink sq512 past512	0.57	0.60	0.56
llama-3.2-1b sq2048	1.45	1.57	1.42
llama-3.1-8b sq2048 (D=128, control)	2.75	2.72	2.72
gpt_oss-win sq512 past8192 w128 (windowed)	n/a	0.13	0.15

gpt-oss-20b at seqlen 16384, interleaved TTFT, 3 rounds — every round of this PR is below every round of #622, against a ~52 ms spread:

arm	mean TTFT	min	max	sd
#622	9,956 ms	9,917	10,016	53
this PR	9,462 ms	9,405	9,504	51

RGP capture, chunk 16, layer 1. The two layer types now dispatch different instantiations, so they separate by name rather than by duration:

dispatch	#622	this PR
full attention	8,313 us (n=13, 8,033-8,613)	5,970 us (n=7, 5,865-6,105)
windowed	229 us (n=13)	233 us (n=6)

Known trade-off

One shape moves the other way. On gpt_oss-20b sq2048 past0 at M_TILES=2 BKV=32, #622's spilled codegen is faster — 3.29 ms against 3.74 pre-window and 3.77 here (medians of 5 interleaved autotuner sweeps). It reproduces, and it does not appear in the other three no-window shapes, where #622 is 20-30% slower at that config. Restoring the pre-window codegen restores its weakness on that shape along with its strengths. Net TTFT still improves by 494 ms, but the ~13% there is unexplained and worth a look on its own.

Related: the regression was partly masked by the autotuner. On #622 the degraded M_TILES=2 BKV=32 loses to M_TILES=1 BKV=32 on three of four no-window shapes, so end-to-end timings understated the per-config damage.

Test plan

• test_gqa_prefill standalone harness, 23 cases, ALL PASS with every relL2 bit-identical to perf(gqa): support sliding windows in the fused flash prefill kernel (-65.7% TTFT cumulative) #622. Removing the empty guard from the no-window path is a behavioural change on paper, so the cases that caught the original NaN were checked specifically: window=1, window > sequence, and past_len 8192.
• Resource usage compared instantiation by instantiation against the pre-window kernel (table above).
• Interleaved harness A/B and interleaved TTFT (tables above).
• RGP dispatch-level capture (table above).
• CI.

All measurements on gfx1151 (Radeon 8060S / Strix Halo), ROCm 7.1.

[github-actions]

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

1. The change is focused. Substantial work links the relevant issue
   or design discussion.
2. The PR documents relevant test results and updates affected
   documentation.
3. If AI tools provided substantial assistance, the description
   explains what was assisted and how it was validated, and commit
   trailers identify the tool. The contributor has reviewed and
   understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

[github-actions]

L2 Accuracy Results (EP vs CPU)

Model	Combined L2	Total Elems	Skipped NaN/Inf
conv_test_hybrid	4.8668E-07	64	0
GroupQueryAttention_seq256	25.2366	2621440	0
MatMulNBits_o_seq128	259.906	368640	0
QMoE_seq128	34.957	368640	0

Threshold: 0.01 | Run: 3528 - Commit: 9615d03

[github-actions]

MorphiZen EP Performance Results

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	7.51	6.35	362	3	1243
GroupQueryAttention_seq128	4313.13	1.67946	11	6	311
matmul_down_seq128	526.15	2.46	75	3	352

EPContext Export Performance

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	7.56	44.28	361	3	15589

EPContext Import Performance

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	7.53	9.24	359	3	15760

OGA Benchmark Results

Model	Warmup	Reps	Prompt Len	Gen Tokens	TTFT (ms)	TPS	Peak Mem (GB)	GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32	1	5	128	128	165.6	79.0	1.33	13.53
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml	1	5	128	128	278.1	40.6	1.22	6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model	TTFT (ms)	TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml	196	40.3

Run: 3528 - Commit: 9615d03

[BoarQing]

http://xsjnts.xilinx.com/ROCm/compare?base_url=http%3A%2F%2Fxsjvitisaijenkins%3A8080%2Fjob%2Fdaily_hip_ep_oga%2F1635&base_job=&base=&curr_url=http%3A%2F%2Fxsjvitisaijenkins%3A8080%2Fjob%2Fdaily_hip_ep_oga%2F1633&curr_job=&curr=

[BoarQing]

acc: http://xsjvitisaijenkins:8080/job/daily_hip_ep_oga/1645/