Skip to content

fix(promote-strided): include hip.npi_* ops in strided-input promotion - #625

Draft
kirankumarmatam-amd wants to merge 2 commits into
mainfrom
fix/qwen-promote-strided-npi
Draft

fix(promote-strided): include hip.npi_* ops in strided-input promotion#625
kirankumarmatam-amd wants to merge 2 commits into
mainfrom
fix/qwen-promote-strided-npi

Conversation

@kirankumarmatam-amd

@kirankumarmatam-amd kirankumarmatam-amd commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Problem

PromoteStridedHipOperands filtered ops using op->getDialect() == HipDialect.
NPI plugin ops (hip.npi_quantize, hip.npi_dequantize, etc.) are registered
in NpiDialect — a separate dialect object — even though their op names start
with hip. and they use the same contiguous-pointer kernel ABI. The dialect
check therefore excluded them, so strided memref.subview inputs (e.g. Q/K/V
slices produced by onnx.Split) were passed directly to NPI kernels that read
them as flat contiguous arrays, producing silently wrong values. On Qwen 7B VLM
(gfx1151) this caused cosine similarity to drop from ~1.0 to ~0.36.

Fix

Replace the dialect-object check with an op-name prefix check
(fullOpName.starts_with("hip.")), which covers both HipDialect ops and
hip.npi_* ops regardless of which dialect object owns them.

Testing

Verified on gfx1151 (Strix Halo, Windows D3D12 HMM): cosine similarity for
Qwen 7B VLM restored to ~1.0 after this fix.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 4.8668E-07 64 0
GroupQueryAttention_seq256 25.2366 2621440 0
MatMulNBits_o_seq128 259.906 368640 0
QMoE_seq128 34.957 368640 0

Threshold: 0.01 | Run: 3535 - Commit: 07eedbc

@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.52 6.20 361 3 1242
GroupQueryAttention_seq128 4406.44 1.69259 12 6 310
matmul_down_seq128 499.02 2.45 74 3 352

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.55 44.05 359 3 15590

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.53 8.66 362 3 15760

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 169.2 79.1 1.33 13.57
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 345.0 40.0 1.22 6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 194 39.3

Run: 3535 - Commit: 07eedbc

Collapse the duplicate comment block into a single coherent explanation
and restore the blank line between the copy-materialization loop and
the dealloc-emission comment that was inadvertently removed.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant