Skip to content

RVV: Optimize conv opt and common opt functions - #4821

Open
typer-J wants to merge 2 commits into
alibaba:masterfrom
typer-J:feature-ConvOpt
Open

RVV: Optimize conv opt and common opt functions#4821
typer-J wants to merge 2 commits into
alibaba:masterfrom
typer-J:feature-ConvOpt

Conversation

@typer-J

@typer-J typer-J commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Supersedes #4457.

This rebases the RVV convolution optimization work onto the latest master and resolves the conflicts called out in the review of #4457.

Changes

  • Add RVV implementations for MNNConvRunForUnitint8_t and MNNConvRunForLineint8_t.
  • Refine RVV depthwise convolution and depthwise deconvolution kernels.
  • Optimize MNNAbsMaxFP32 across the realSize dimension.
  • Refine the RVV matrix add, subtract, and product kernels.
  • Keep the common C-linkage ConvOpt helpers on RVV; exclude only the two int8 helpers that have matching RVV replacements.

Rebase conflict resolution

Earlier verification (before this update)

  • clang-format --dry-run --Werror passed for all eight changed RVV source files.
  • LLVM 22 successfully compiled all changed RVV kernels in syntax-only mode for rv64gcv (--target=riscv64-unknown-linux-gnu -march=rv64gcv -mabi=lp64d).
  • WSL Release host build with MNN_BUILD_TEST=ON completed successfully.
  • Full host unit suite passed: 380 passed, 0 failed, 0 skipped.

AbsMax regression repair (2026-09-10)

For realSize=1, vectorizing across positions leaves only one active lane and performs repeated strided loads and output read/modify/write operations. Reduce contiguous channels in registers instead, preserving inactive accumulator lanes on the final tail and reducing once at the end. The C4 runtime layout is unchanged.

On native RVV VLEN=128 (GCC 14.3.1), six independent alternating-order runs measured:

channels realSize previous kernel median ns repaired median ns speedup over previous
1024 1 2283.66 142.06 16.08x
4096 1 9273.88 560.40 16.55x

Across 60 shapes, no >5% slowdown against the previous PR kernel; all median speedups against the registered scalar baseline exceed 1 (minimum 1.36x). Scalar timings vary substantially; previous/new RVV timings are stable. These are kernel measurements, not model throughput claims.

Validation for this update:

  • Native 1/4-worker independent scalar-oracle test: 18,576 bitwise comparisons per worker, including empty dimensions, channel tails, packs 4/8/16, signed zero, all-NaN, infinity and output guards.
  • QEMU VLEN 128/256/512/1024, each with 1/4 workers: passed (GCC 15; correctness only).
  • 936 scalar/previous/repaired comparisons per native benchmark process, six processes: passed.
  • Full native RVV Release static library and test executable: built with Ninja, low-memory enabled, fast-math disabled. Narrowed the original ConvOpt exclusion to the two matching int8 replacements to fix missing common C-linkage symbols.
  • Qwen2.5-0.5B fixed-trajectory short/long logits: byte-identical before/after (7 steps to EOS / 16 steps). Both builds retain the same link repair; only AbsMax differs.
  • Deconvolution (3 tests) and matmul (2 tests): passed with both 1 and 4 threads.
  • Known existing failure: op/convolution/weighti8i4conv2d INT4 async case fails with the same error in previous AbsMax, repaired AbsMax, and existing RVV baseline, with both 1 and 4 threads. The convolution group has 5 passes and 1 failure in each thread setting. This update does not claim a fully passing regression suite.

Standalone reproduction: test/backend/cpu/RVVAbsMaxTest.md.

Module

CPU/RVV

Type

  • Feature
  • Perf
  • Bugfix
  • Refact
  • Style
  • Doc
  • Test
  • Chore

Checklist

  • Commit message follows [Module:Type] Description format
  • Code compiles without errors
  • Full suite passes (known baseline INT4 async failure documented above)
  • No unrelated changes included

Add and refine RVV implementations for ConvOpt depthwise,
int8, deconv, matrix, and related low-memory helper kernels.

Co-authored-by: Sherlockzhangjinge <zjgzhangjinge@outlook.com>
Co-authored-by: jxgxxx <1955992348@qq.com>
Co-authored-by: lyd1992 <liuyudong@iscas.ac.cn>

Signed-off-by: typer-J <2236066784@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant