Skip to content

feat(hipsr): add pool-alloc greedy grouping - #605

Merged
zz002 merged 1 commit into
mainfrom
feat/hipsr-pool-alloc-110-grouping
Jul 31, 2026
Merged

feat(hipsr): add pool-alloc greedy grouping#605
zz002 merged 1 commit into
mainfrom
feat/hipsr-pool-alloc-110-grouping

Conversation

@zz002

@zz002 zz002 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

-hipsr-pool-alloc now groups each pool domain's allocs by lifetime: an alloc joins the first group whose members are all disjoint from it, otherwise it opens a new one. The pass still rewrites nothing; the emit-pool-report remark gained the group each alloc landed in.

Related issue or design

Upstream wcy123/onnx-hipdnn-ep#110 (greedy grouping utility), a step of wcy123/onnx-hipdnn-ep#19, which links the HIPSR Pool Allocation Pass design page. Follows #591 (#109, liveness).

Why

Group numbering ends up in the report, and #117 will turn it into pool offsets, so it has to be reproducible. Two things make it so: allocs are collected by walking the block rather than iterating the Value -> Lifetime map, whose order is pointer-hash dependent; and the sort by lifetime start is stable, because allocs feeding several outs of one DPS op share a start and an unstable sort would order them arbitrarily.

Touching endpoints count as overlapping. The op that first writes the later buffer is still reading the earlier one, so [1,3] and [3,5] cannot share space.

What

  • greedyGrouping(Block &, const DenseMap<Value, Lifetime> &) returning groups as vectors of allocs, next to computeLiveness in the same anonymous namespace.
  • emitPoolingReport takes the groups and reports lifetime [a,b] group N. Per-domain totals (alloc and group counts, reuse ratio) belong to #118.
  • pool_alloc_report.mlir: existing ranges gained their group, plus the two boundary topologies.

Test plan

  • Full check-hip-mlir-lit — 361 passed, 3 unsupported, 0 failed.
  • Mutation check: flipping one expected group N fails the run, so the new assertions are not vacuous.
  • pre-commit run --all-files clean.

Grouping is covered by chromatic number of the interval graph: coalesce_static (four disjoint allocs of differing sizes, one group), interleaved_allocs and hoisted_allocs (one alloc over two disjoint ones, two groups), split_three_groups (pairwise overlap, three groups).

Notes for reviewers

  • coalesce_static and split_three_groups are lifted verbatim from the reference implementation in #572, minus its CHECK lines, which assert the pooled IR that does not exist yet. Keeping the fixtures identical means #117 adds assertions instead of rewriting tests. They live in pool_alloc_report.mlir for now because a remark is the only thing to assert before the rewrite is wired; #117 moves them to pool_alloc.mlir.
  • Tests are lit rather than gtest: the inputs are a Block and its Values, which gtest would have to build by hand, and the RUN line's --implicit-check-not=hipsr.get_pool --implicit-check-not=memref.view doubles as proof the pass is still a no-op. Happy to add a unit test if reviewers prefer one.
  • greedyGrouping runs unconditionally and its result is only read under emit-pool-report, matching how computeLiveness landed in #109. #117 consumes it for real.

Checklist

  • The change is focused, or links a design/series explaining its scope.
  • Relevant tests were added or updated and the results are documented.
  • User-facing or design documentation was updated when needed.
  • Substantial AI assistance is disclosed, and I reviewed and understand the result.

AI assistance: Cursor implemented greedyGrouping, ported the two fixtures, and ran the lit and pre-commit validation above; I reviewed the result and hand-checked every expected lifetime and group against the fixture IR.

Place each alloc into the first group whose members all have lifetimes
disjoint from it, opening a new group when none fits. Touching endpoints
count as overlapping: the op that first writes the later buffer is still
reading the earlier one.

Allocs are collected in block order and stable-sorted by lifetime start,
so group numbering does not depend on DenseMap iteration order.

Grouping stays analysis only; the pass still rewrites nothing. The
emit-pool-report remark now reports "lifetime [a,b] group N".
@github-actions

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

@github-actions

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 4.8668E-07 64 0
GroupQueryAttention_seq256 25.2366 2621440 0
MatMulNBits_o_seq128 259.906 368640 0
QMoE_seq128 34.9552 368640 0

Threshold: 0.01 | Run: 3452 - Commit: 3691f44

@github-actions

Copy link
Copy Markdown

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.51 6.26 367 3 1243
GroupQueryAttention_seq128 4391.36 1.68884 11 6 310
matmul_down_seq128 526.74 2.38 74 3 351

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.52 46.67 361 3 15590

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.51 9.77 359 3 15760

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 175.0 79.4 1.33 13.54
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 226.5 40.8 1.22 6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 202 39.6

Run: 3452 - Commit: 3691f44

@wcy123 wcy123 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zz002
zz002 merged commit 2f4cccf into main Jul 31, 2026
6 checks passed
@zz002
zz002 deleted the feat/hipsr-pool-alloc-110-grouping branch July 31, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants