[CI]【Hackathon 10th Spring No.53】headwise SWA unit test [cf]

[ghost]

Motivation

Modifications

Usage or Command

Accuracy Tests

Checklist

• I have submitted the CLA (only first PR)
• My PR title follows the convention
• My changes pass all tests

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

cloudforge1 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[paddle-bot]

Thanks for your contribution!

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | pr_review | 2026-05-03 01:36:09

📋 Review 摘要

PR 概述：PR 标题声称新增 headwise SWA 单测，但实际 diff 仅在 README_EN.md 末尾增加了一个空行，未见任何单测代码。
变更范围：README_EN.md（文档）
影响面 Tag：[Docs]

📝 PR 规范检查

PR 规范存在以下问题：

1. Tag 与 diff 不符：标题使用 [CI] 但实际变更文件为 README_EN.md，应使用 [Docs]（architecture.md §8：README* → [Docs]）
2. 标题含非标准后缀：[cf] 不在官方 Tag 列表中，应删除
3. 描述模板未填写：Motivation / Modifications / Usage or Command / Accuracy Tests 均为 TODO 占位符
4. Checklist 结构不符：当前 Checklist 与 §D2 模板不一致，需替换为规范版本

标题建议（可直接复制）：

• [Docs] Add trailing newline to README_EN.md

PR 描述建议（可直接复制，必须复刻 checklist §D2 模板的完整结构）：

## Motivation
在 README_EN.md 末尾补充空行，符合文件末尾换行规范（trailing newline）。

## Modifications
- `README_EN.md`：末尾新增一个空行

## Usage or Command
N/A

## Accuracy Tests
N/A

## Checklist

- [x] Add at least a tag in the PR title.
  - Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
  - You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.

问题

级别	文件	概述
📝 PR 规范	—	Tag [CI] 与实际变更（README 文档）不符，应为 [Docs]；标题含非标准后缀 [cf]；描述模板各段均为 TODO 占位符；Checklist 结构不符合 §D2 要求

总体评价

本 PR 实际 diff 仅在 README_EN.md 末尾新增一个空行，与标题声称的"headwise SWA unit test"严重不符，请确认是否提交了完整内容。PR 规范问题需按建议修正后再合入。

[PaddlePaddle-bot]

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-05-03 02:41:09

CI报告基于以下代码生成（30分钟更新一次）:

• PR commit: fb18001
• Merge base: d70f33d (branch: develop)
• 查看完整 Diff
• CI 详情

---

1 任务总览

所有已执行任务均通过，无失败任务。✅

总执行（rerun次数）	总任务	✅ 通过	❌ 失败	⏳ 运行中	⏸️ 等待中	跳过
27(0)	27	10	0	0	0	17

⚠️ 注意：以下 7 个 Workflow 处于 action_required 状态（等待审批后才会执行）：Check PR Template、Codestyle-Check、CI_HPU、ILUVATAR-CI、Approval、PR Build and Test、CI_XPU。这些 Workflow 需人工审批触发。

注意：action_required workflows 不计入上表的任务统计。

---

2 任务状态汇总

2.1 Required任务 : 未配置

当前分支未配置 Branch Protection Rules 中的 Required 状态检查，或 API 权限不足无法获取。所有已执行任务均为可选任务。

2.2 可选任务 — 10/10 通过

可选任务不阻塞合并，失败仅供参考。

状态	任务	耗时	日志	重跑
✅	10 个可选任务通过（Trigger Jenkins for PR × 10，来自 CI_METAX）	-	-	-
⏭️	17 个任务已跳过（cherry-pick × 17）	-	-	-

---

3 失败详情（仅 required）

无 required 失败任务。

[boby-cloudforge]

PR1 Body — [Feature][KVCache] Support head-wise SWA cache recycle in ResourceManagerV1 (Hackathon 10th Spring No.53)

Draft for review before opening the PR. Saved here, not in /tmp.
5 required sections per FastDeploy CI gate. Word budget ≤600.

---

Motivation

Hackathon 10th Spring Task No.53 — 离散 KV Cache 管理和 AppendAttention 算子的性能优化 (PR1 of 2). Spec: https://github.com/PaddlePaddle/community/blob/master/hackathon/hackathon_10th/【Hackathon_10th】开源贡献个人挑战赛春节特别季—任务合集.md#no53.

For models that mix Sliding-Window Attention (SWA) heads with full-attention heads inside the same layer, today's V1 KV-cache scheduling path (ResourceManagerV1 + PrefixCacheManager, gated by the default-on ENABLE_V1_KVCACHE_SCHEDULER=1) allocates one shared block_idx per layer for all heads. SWA heads finish their window long before full-attn heads, but their cache stays pinned until the whole layer evicts. Throughput suffers.

This PR teaches the V1 scheduler + PrefixCacheManager to manage block_idx per head (head-wise SWA layout) and recycle a SWA head's cache as soon as it crosses its window — the per-head equivalent of what PR #6702 did for V0.

Authorship: this PR is independently designed and implemented by the submitter for Hackathon 10th Spring No.53. The earlier community PR #6702 (V0, not merged) is referenced as prior art only; no code is lifted unattributed. Any future contributor work will be acknowledged via per-commit Co-authored-by trailers.

RFC: PaddlePaddle/community#1364.

⚠️ DRAFT — opened early to surface CI feedback while commits 2/4–4/4 land. Commit 1/4 (this push) is foundation only: 3 default-off env flags + ERNIE SWA fixture hook (22 LOC, no behavior change at default).

Modifications

Area	Change
fastdeploy/cache_manager/prefix_cache_manager.py	Per-request head-wise GPU free list (gpu_free_block_list_head_wise[head]); allocate_gpu_blocks_head_wise / recycle_gpu_blocks_head_wise; TP-aware sizing (num_key_value_heads // tp_size)
fastdeploy/engine/sched/resource_manager_v1.py	recycle_request_swa_head_cache (per-head cursor advance ≥ window+sink); _should_skip_swa_recycle_for_overlap (per-request cache_swap_metadata / cache_evict_metadata inspection); P4 cleanup in _free_blocks
fastdeploy/model_executor/models/paddleformers/base.py	Default-off ERNIE SWA fixture (window/sink/skip-freq/ratio) gated by FD_T53_HEAD_WISE_SWA_FIXTURE=1
fastdeploy/config.py	+20 — Engine-main FDConfig fixture: mirror the paddleformers/base.py head-wise SWA attribute injection so ResourceManagerV1._should_use_head_wise_swa (engine-main) sees the same model_config.head_wise_swa_ratio as the worker. Gated on FD_T53_HEAD_WISE_SWA_FIXTURE.
Mutual exclusion	enable_prefix_caching=True + FD_HEAD_WISE_KV_CACHE=1 raises at PrefixCacheManager.__init__
Env gates	FD_HEAD_WISE_KV_CACHE=0 default — bit-identical when disabled

Tests use real lightweight objects + object.__new__/AST or shape oracles (no MagicMock-only). PR2, not PR1, owns kernel-visible block_tables_headwise / FP8 scale-layout changes.

PR2 (separate) lands the AppendAttention rank-2 block_tables_headwise ABI + ForwardMeta wiring + kv_num_heads field as a frozen-shape parameter; PR1 keeps share_inputs.block_tables 2D and reaches the +30% recycle gate via cache-manager-side changes only.

Usage or Command

# Enable head-wise V1 cache + timely SWA recycle.
# All four env vars must be set together — partial activation is silently a no-op.
# Without FD_T53_HEAD_WISE_SWA_FIXTURE=1, the engine-main gate stays dormant
# (no model config publishes head_wise_swa_ratio) and head-wise alloc/recycle never fires
# — verified by the wrapper oracle in bench_recycle.sh.
export FD_T53_HEAD_WISE_SWA_FIXTURE=1     # engine-main FDConfig fixture (config.py)
export ENABLE_V1_KVCACHE_SCHEDULER=1      # default; shown for clarity
export FD_HEAD_WISE_KV_CACHE=1            # enables per-head block tables
export FD_T53_HEAD_WISE_SWA_RATIO=1.0     # SWA recycle ratio (>0 = recycle active)
python -m fastdeploy.entrypoints.openai.api_server \
    --model baidu/ERNIE-4.5-21B-A3B-Paddle \
    --max-model-len 32768

Accuracy Tests

Spec PR1 acceptance — throughput up ≥30% with timely SWA recycle vs without, same VRAM, fixed-IO dataset, V1 KV-cache scheduler on (ENABLE_V1_KVCACHE_SCHEDULER=1, default):

Round 2 (gate run — 128 prompts):

Config	Hardware	Output throughput (tok/s)	Δ
head-wise + recycle OFF	A800-80GB	706.29	baseline
head-wise + recycle ON	A800-80GB	1107.98	+56.9% ≥30 ✓

Round 3 (full run — 1024 prompts):

Config	Hardware	Output throughput (tok/s)	Δ
head-wise + recycle OFF	A800-80GB	722.93	baseline
head-wise + recycle ON	A800-80GB	1270.87	+75.8% ≥30 ✓

Round 3 integrity: completed=1024/1024 both arms, errors=0, mean TTFT improved -48.0% (2,708 s → 1,407 s).

Benchmark: FastDeploy/benchmarks/benchmark_serving.py — random fixed-IO dataset, input≈10.6k tokens avg / output≈4k tokens avg, request-rate=8, seed=42, --ignore-eos, server --max-concurrency=8192, YAML eb45-21b-a3b-32k-bf16-kv50-512s.yaml (kv_cache_max_ratio=0.50, max_seq_len=512). Fixed-IO integrity: both arms produce identical total_input_tokens=1,356,656 / total_output_tokens=518,946 for the 128-prompt gate run. Round 2 harness gate: completed=128, nonempty_errors=0. Round 3 target: completed=1024.

Hardware note for reviewers: spec does not pin PR1 hardware. Numbers above are A800-80GB (SM80) via Baidu AI Studio. If H/B card access is granted (cc @luotao1), we will append H/B numbers as supplementary evidence. PR2 (5% TTFT/TBT) does require H/B per spec; tracked separately.

Correctness:

• CPU pytest coverage under tests/cache_manager/test_head_wise_*.py, tests/cache_manager/test_swa_recycle*.py, and tests/layers/test_append_attention_head_wise_shapes.py — real _FakeCacheManager + object.__new__(ResourceManagerV1) + AST/shape oracles. No MagicMock-only tests.
• A800 smoke (bsz=4, seq=1024) + long-context recycle smoke — TBD, pending CI access
• GSM8K parity (head-wise vs non-head-wise abs diff ≤ 0.5 pp) — TBD, deferred to follow-up validation pass

CI run:

Checklist

• pre-commit run --all-files clean
• All CI checks green (Coverage / base_tests / codestyle / iluvatar / xpu)
• Reviewer-requested changes addressed
• No prohibited claims in PR body (verified by pre-push grep): "first in framework", "novel research", "unique to FastDeploy"
• Authorship statement accurate (no unattributed lifted code)
• Hardware label on every benchmark number matches the actual card used

[cloudforge1]

Closing — superseded by #7717 (PR1) and #7718 (PR2) which are the latest v4 versions under active review. Please ignore this PR.