feat: support Qwen3.5 Decode Context Parallel on Ascend NPU. by Enguikong · Pull Request #2125 · xLLM-AI/xllm

Enguikong · 2026-08-05T03:31:36Z

Qwen3.5 Decode Context Parallel（DCP）Ascend NPU 适配

概要

本 PR 为 Ascend 910C 上的 Qwen3.5 GQA 模型增加 Decode Context Parallel（DCP）支持。DCP 在既有 Tensor Parallel 组内沿 decode context/KV 维度分片，不增加 world size。

当前范围包括：

Qwen3.5 dense 与 MoE。
normal prefill 与 decode。
通过 --enable_experimental_dcp_chunked_prefill=true 显式启用的 chunked prefill。
使用 DCP-local block table 与 sequence length 的 prefix cache。
用户请求 mixed batching 时的 scheduler 安全处理。
eager schedule overlap。
DCP decode 的 ACLGraph capture/replay。
已验证 TP4/DCP2 与 TP8/DCP2 拓扑。

当前分支 HEAD：93830fdd。

当前 PR 状态：

已合并最新 upstream base 9e0f1402c1cee9798a485a45913296f83ebd1ec3。
最新一轮 spawn-worker protocol 冲突已完成代码级合并：保留 upstream draft_sampling_mode，并将 decode_context_parallel_size 追加到协议尾部；父进程 argv、子进程 parser、server 构造与协议测试保持一致。
合并后官方 NPU build 与冲突相关定向回归已通过；与当前 base 的代码冲突已解决，PR 已是 Ready for Review，正在等待新一轮必需检查与审批。
当前真机验证范围为 DCP2：TP4/DCP2 与 TP8/DCP2。更大的 DCP size 不在本 PR 已完成的真机声明中。

背景与动机

Qwen3.5 使用 GQA，并包含 GDN 层。该模型的 prefix cache 依赖 chunked prefill 保存 linear-state checkpoint，因此 DCP 必须同时正确处理 decode、chunked prefill、prefix cache 及其 metadata。

DCP 的首要价值是长上下文 attention/KV ownership 分片。它会增加 query、partial output 和 LSE collective，因此本 PR 不承诺 eager DCP 在短上下文下必然降低延迟或提升吞吐。当前 scheduler 仍按 global sequence length 预留 KV cache，物理 HBM 1/dcp 容量收益不在本 PR 范围内。

设计

DCP 拓扑与 KV ownership

在 TP ranks 内建立 DCP/KV-replica process group。
按 block-size 粒度交错 virtual KV blocks。
从 global metadata 计算每个 rank 的 local KV sequence lengths 与 local block table。
owner block 保留原 physical block ID。

Attention 路径

每个 rank 对本地 KV shard 执行 FIA。
gather 所需的 query/head group。
gather partial attention output 与 LSE。
使用 fp32 online-softmax 公式合并各 rank partial。
decode、chunked prefill、prefix cache 与 multi-sequence batch 共用 DCP-local metadata 契约。

Chunked prefill 安全门

新增 --enable_experimental_dcp_chunked_prefill，默认 false。
gate 使用 effective chunked-prefill 状态，包括 multi_slo_and_prio 的隐式启用。
P/D 与 speculative 等未支持组合继续独立 fail-closed，不会被 experimental flag 绕过。

Mixed-batch 安全

当 decode_context_parallel_size > 1 时，即使用户保留 --enable_mix_batch=true，scheduler 也会把实际 batch mode 解析为 mix=false。这样不修改原 flag 的非 DCP 语义，同时避免 MIXED batch 进入 DCP worker 路径。

Schedule overlap

DCP 的 per-step metadata 受现有 scheduler/worker FIFO 与 stream ordering 保护。完成生命周期审计及 overlap-on/off 上卡对照后，本 PR 放开 eager schedule overlap。

ACLGraph

graph 路径使用稳定的 capture/replay 存储，避免引用临时 tensor：

caller-owned FIA output、softmax LSE 与 workspace。
FiaGraphTask 在 replay 前更新本步标量参数。
persistent global/DCP-local block-table buffer。
persistent local indices；local-table selection 在 captured graph 外执行。
metadata update 与 graph replay 之间使用 event/barrier 定序。

上卡迭代中定位并修复了两类 capture 生命周期问题：FIA wrapper 的临时 workspace，以及被 GatherV3 捕获的临时 arange index tensor。

正确性契约

本 PR 不把跨 DCP 的逐 token bitwise exact 作为发布契约。FIA partial output 在 fp32 merge 前已经以 BF16/FP16 落值；同时 NPU greedy generation 在 DCP1 基线上也存在自然文本 run-to-run 漂移。

因此采用以下验收方式：

structural 与 multi-sequence suite 精确匹配 golden。
natural text 完整保留用于审计，但不进入 exact suite hash。
GSM8K 与 C-Eval 全量下游精度。
候选相对配对基线不劣于 1 percentage point。
同时报告配对 McNemar 检验。
HTTP/prediction errors、cache 契约、FATAL/CHECK、设备故障、HCCL timeout 与进程回收继续作为硬门。

验证结果

离线验证

最新 upstream 合并后官方 NPU build：PASS；最终 binary SHA256 为 1f3253c7bc2091f21c5437616b7cf16f8515e4f32860ca13c2577cdd0b42e211。
冲突文件 clang-format：PASS；相对 upstream 的净 diff check：PASS；无冲突标记与 _Exit 修改。
dcp_compat_test：19/19 PASS。
scheduler DCP/mixed/effective-chunked 定向回归：6/6 PASS。
config_json_test：22/22 PASS。
spawn_worker_protocol_test：6/6 PASS。
python_executor_test：102 passed、2 skipped；完整测试目标单独执行 PASS。
xllm_export_import_test：PASS；直接加载构建产物并验证 LLMMaster、AttentionMetadataView 与 StateDict 可用。
cp_group_ranks_test：14/14 PASS。
npu_dcp_attention_utils_test：16/16 PASS。
npu_dcp_attention_test：2/2 PASS。
acl_graph_executor_test 的 DCP/冲突相关过滤集：4/4 PASS。
acl_graph_task_update_test：4/4 PASS。

最新 PR 收敛清理包含 a7ce44e0、ed74e9bf、95dda04e 三个提交，相对当时 upstream merge base 共 10 个文件、净 +62/-132：删除不可达的非 LLM DCP 透传与生产测试 oracle，恢复 /v1/completions 上游行为，测试改为链接既有 DCP attention utils，并将 ACLGraph replay barrier 精确收窄到存在 FIA graph task 的路径。清理后官方 build、上述 16/14/19 单测及格式检查均通过。

最新 x86_64 NPU CI 暴露了 upstream/main 同样存在的 GLM5.2 回归：共享 DeepseekV3MoE 已采用 lazy expert-weight allocation，但 GLM5.2 loader 与测试仍假定构造阶段已分配权重。该测试在脱离并行 CTest 后仍稳定失败，因此没有通过把 python_executor_test 改为串行来掩盖问题；提交 7ee62fc1 让 GLM5.2 loader 复用既有分阶段分配/格式化流程，并补齐对应回归断言。

修复 GLM5.2 后，wheel 导入又暴露出独立的 Python bridge 生命周期问题：原先两个 PYBIND11_EMBEDDED_MODULE 静态构造器会在 Python 解释器已初始化后调用 PyImport_AppendInittab，触发 Can't add new modules after the interpreter has been initialized。提交 e2012809 改为在当前解释器中按需创建内部模块并注册到 sys.modules，同时保留 wheel 后续使用 Python model backend 的能力；没有采用仅让 native binary 链接通过、却破坏 wheel Python backend 的规避方案。官方 build、python_executor_test、新增的 wheel 导入回归、build tree 导入和隔离 wheel 安装导入均已通过，最终扩展中也不再包含 PyImport_AppendInittab 或旧 embedded-module 初始化符号。

随后 x86_64 NPU CI 的 1417 项并行测试、wheel 构建与安装均已通过，但最终 offline smoke 仍使用已经从 CLI 删除的 --devices="npu:7"，因此在参数解析阶段退出。该问题来自上游移除 --devices 后遗漏的 workflow 调用点，与 DCP 和测试并行无关；提交 93830fdd 将其改为项目文档约定的 ASCEND_RT_VISIBLE_DEVICES=7，不修改测试顺序或推理逻辑。

完整 acl_graph_executor_test 还复现了 upstream 当前代码与用例共有的 GenericSpecVerifyCaptureKeepsPersistentBlockTableWidth 容量断言失败；该用例及触发实现相对 upstream/main 无净差异，DCP 专属 DcpDecodeUsesStableRankLocalBlockTable 已独立通过，因此未在本 PR 中夹带无关修复。

上卡功能验证

Phase E：DCP2 + chunked prefill + prefix cache，用户侧 enable_mix_batch=true；并发 decode/prefill 注入完成，无 MIXED FATAL。
Phase F：dcp{1,2} × prefix{off,on} × chunk{128,256} 8/8 完成，prefix cache 真实命中，服务干净停服。
eager schedule overlap：目标用例中 overlap-on 与 overlap-off 的 DCP2 输出逐 token 对齐。
ACLGraph：短/长上下文、变化的 local KV lengths、连续 replay 与 concurrency 64 均完成，无 507011、GatherV3 越界或 HCCL timeout。
F4 graph 回归：DCP1 graph 与 DCP2 graph 各 5/5 case 正常生成并干净停服；随后完成 20×5×256 token soak，保留的 rank 日志中 507011、GatherV3 与 FATAL 均为 0 命中。

prefix-cache 契约由内部 metadata、runner 与服务日志证据验证。本 PR 不在 /v1/completions 响应中新增 cached_tokens 字段；该 API 可观测性属于独立 follow-up，不能把该字段作为本 PR 已实现或已验证的接口能力。

大模型 EvalScope

配置：

Qwen3.5-9B，TP8。
Qwen3.5-35B-A3B，TP4。
模式：DCP1 eager、DCP2 eager、DCP2 graph。
GSM8K 全量 1319 题。
C-Eval 全量 1346 题。
greedy generation，关闭 thinking，最大输出 4096。
正式 client/server concurrency 32，另补 concurrency 64 压力验证。

模型 / 数据集	DCP1 eager	DCP2 eager	DCP2 graph	结果
9B GSM8K	0.9553	0.9484	0.9416	PASS
9B C-Eval	0.8462	0.8462	0.8425	PASS
35B-A3B GSM8K	0.9674	0.9636	0.9704	PASS
35B-A3B C-Eval	0.8989	0.8952	0.8967	PASS

八个配对比较全部通过 1pp 非劣门；所有配对 McNemar p > 0.05；正式 run prediction error 为 0。

concurrency 64 使用 GSM8K 256 题 canary 与 C-Eval 1346 题全量复核，未发现 graph 独有的精度退化、请求错误或设备故障。

最终大模型矩阵中：

16/16 service run 停服成功。
96/96 rank 进程记录为 terminated。
未发现 glog FATAL/CHECK、507011、GatherV3 越界或 HCCL timeout。

性能观察

以下是自然数据集观测，不是最终定长 benchmark：

DCP2 eager 相对 DCP1 eager 的 aggregate output TPS 下降约 5%–13%，与新增 DCP collective 的方向一致。
DCP2 graph 相对 DCP2 eager：9B aggregate output TPS 提升约 7%–17%，35B-A3B 提升约 30%–36%。
在现有 9B 负载上，graph 大致回收 eager DCP 开销；在现有 35B 负载上，DCP2 graph 的 aggregate output TPS 高于 DCP1 eager。

自然数据集在不同模式下可能生成不同长度，因此固定 input/output 的 TTFT、TPOT/ITL、QPS、aggregate output TPS 与 HBM benchmark 仍需继续补齐。本 PR 转为 Ready for Review，但在该证据完成并复核前不声明稳定性能收益。

2026-08-17 的第一轮 9B/35B 定长矩阵没有产生可用性能 case，已明确判为无效证据，不纳入本 PR 结论。根因是性能 runner 的控制流：strict structural readiness 提前返回后未清理当前服务，同时 Bash 条件函数中的 set -e 语义使失败的 card preflight 没有阻止下一 mode 启动，造成服务重叠和 35B graph OOM。该 OOM 属于测试污染，不是 DCP graph 结论。

2026-08-20 已完成 runner 离线修复：

性能 readiness 默认按 HTTP contract 判定，非 200、空 case、超时及非预期错误继续 fail-closed。
所有正常/异常路径均执行 exact-stop 与 poststop 核卡。
preflight、stop、poststop 或 service-exit 等基础设施失败会中止整个 matrix，不再启动下一 mode。
runner 与 fixed-serving 工具离线测试合计 9/9 PASS。
修复后 runner SHA：c9ff9c609ef5fd31239ed80ae03b95fa334abe185c912fdb31e896ebdc2da887。

修复后的 35B-A3B 三模式 canary 已完成两轮，其中第二轮使用连续 Phy 8,9,10,11：

两轮共 18/18 mode-case PASS，156/156 measured request HTTP 200 且生成长度精确。
readiness、exact-stop、poststop 核卡全部通过；6/6 stop 成功、24/24 rank 精确终止。
未发现 glog FATAL/CHECK、OOM、507011、GatherV3 越界或 HCCL timeout。
DCP2 eager 相对 DCP1 的 output TPS 在三个 canary case 为约 0.73x–0.88x；DCP2 graph 相对 DCP2 eager 在 1K/c4 与 8K/c16 为约 0.77x–0.87x，在 32K/c1 为约 1.49x–1.53x。该方向在两轮中复现，表明 graph 收益依赖上下文和并发，短请求/高并发退化需要在最终候选上继续 profiler 与正式矩阵调查。

上述 canary 绑定冲突前 HEAD 3844bd5，只证明 runner 方法学与合并前性能基线，不冒充最终候选性能证据。当前 93830fdd 已完成最新 upstream 合并、官方构建、协议/DCP 兼容定向回归、Python executor 回归修复、wheel 导入生命周期修复及 NPU wheel smoke 参数迁移；清理栈另已完成 DCP1/DCP2 graph 回归及 20×5×256 token soak。后续仍需在最终候选上完成 9B/35B 的 Core/Full 定长性能、EvalScope 回归与 DCP profiler。正式定长矩阵覆盖 input 1K/8K/32K/64K、output 256/1024、concurrency 1/16/64。

已知限制

FIA partial 在 BF16/FP16 落值后再 fp32 merge，因此 DCP1 与 DCP2 不保证 bitwise token exact。
Qwen3.5-35B-A3B 的 teacher-forced 实验中，观察到某一步约 0.044 的 log-probability 差。完整 GSM8K/C-Eval 未显示统计显著的下游退化，但本 PR 主动披露该数值敏感性。
scheduler KV allocation 仍按 global sequence length 申请。本 PR 已实现 ownership/compute 分片，但尚不声明每 rank KV HBM 按 1/dcp 下降。
disaggregated P/D 与 P/D speculative decode 继续 fail-closed。
本 PR 不支持 Ascend 950。
独立的 NPU 停服 _Exit 修复不在本分支，见 PR bugfix: clean NPU process shutdown (bypass process-static destructors). #2209。
/v1/completions 暂不暴露 prefix-cache cached_tokens；该 API 可观测性将作为独立后续改动处理。

不包含的范围

Prefill Context Parallel。
P/D DCP KV transport。
P/D speculative decode。
scheduler local KV allocation / HBM capacity reduction。
独立 NPU shutdown _Exit 修改。

Reviewer 关注点

DCP chunked prefill 保持 experimental opt-in，并在启动时明确 WARNING。
mixed batching 只在 DCP 配置中自动解析为关闭，原用户 flag 在非 DCP 配置中的行为不变。
graph 修复通过消除 captured temporary-address dependency 解决问题，而不是增加 KV length bucket workaround。
PR 已转为 Ready for Review；定长性能矩阵完成前不把 DCP 描述为无条件性能优化。

Shard standard attention KV cache along sequence within a TP group during decode, reusing TP cards without expanding world size. GDN layers and prefill are untouched, aligning functionally with vllm-ascend DCP. - DCP-0a/0b: independent decode_context_parallel_size flag, startup GQA-topology validation, and TP-internal KV-replica subgroup (dcp_rank = tp_rank % dcp). - DCP-1c: owner-mask cache-slot remap keeping original physical slots (parallel_state::remap_dcp_cache_slots, integer floor_divide) + local block table selection by original allocator id. - DCP-2: decode via FIA with softmax_lse, zero-shard normalization, all-gather partials, fp32 online-softmax merge. - First-version startup compat gates (dcp_compat.h): fail-closed on chunked prefill, prefix cache, schedule overlap, P/D, speculative, and unvalidated MoE. Wording is "does not yet support", not "incompatible". - Tests: cp_group_ranks (incl. floor_divide regression), dcp_compat (12), fia_decode_lse probe, dcp_attention. Validated: Qwen3.5-2B tp=4/dcp=2 dense, per-token aligned with dcp=1 across short/boundary/L513/multi-sequence cases. MoE and HBM savings are follow-ups. Note: committed with --no-verify; pre-commit clang-format hook could not run (container virtualenv broken / physical host lacks pre-commit). All staged C/C++ verified clean via clang-format --dry-run --Werror manually.

Move compute_dcp_local_kv_seq_lens and merge_dcp_partials out of attention.cpp into layers/npu_torch/dcp_attention_utils.{h,cpp} (namespace xllm::layer::detail). These are the production DCP-2 local-length and online-softmax merge implementations — attention.cpp now calls detail::* rather than defining them inline. The extraction makes them linkable from a CPU-only float64 unit test (dcp_attention_utils_test) that verifies sharded-KV merge equals full attention at 1e-10, invalid/all-invalid LSE shards, and dcp=4 tail allocation — without pulling in torch_npu / libpython. Note: committed with --no-verify; pre-commit clang-format hook cannot run (container virtualenv broken). All 4 C/C++ files verified clean via clang-format --dry-run --Werror manually.

DCP reuses TP cards, so cp_size can be 1 while decode_context_parallel_size is > 1. resolve_batch_mode only closed mixed batching for cp_size > 1, leaving DCP + chunked prefill to build a MIXED batch that trips the worker CHECK and FATALs the process under concurrency. Thread decode_context_parallel_size into ContinuousScheduler::Options (default 1) from every scheduler construction site and force enable_mix_batch=false when it is > 1. The worker CHECK is kept as a second layer of defense. Co-Authored-By: Claude <noreply@anthropic.com>

Add --enable_experimental_dcp_chunked_prefill (default false), wired through ParallelConfig (flags, JSON, config dump, option category), Options, and xllm.cpp. When decode_context_parallel_size > 1 with chunked prefill enabled, startup validation now rejects unless the experimental flag is set, and the master logs a one-time warning noting the non-bitwise-equivalence, automatic mixed-batch closure, and rollback. The experimental opt-in only lifts the chunked prefill rejection; it does not bypass the schedule-overlap, disaggregated-PD, speculative, or MoE model-type rejections. Rework dcp_compat_test into the full experimental matrix and correct the attention merge comment to describe the partial-output quantization as a few 1e-3 (measured max 0.00294) rather than ~1e-3. Co-Authored-By: Claude <noreply@anthropic.com>

priority_strategy=multi_slo_and_prio forces chunked prefill in the scheduler regardless of the raw enable_chunked_prefill flag. The DCP experimental gate and startup warning only checked the raw flag, so a service with decode_context_parallel_size>1, enable_chunked_prefill=false and multi_slo_and_prio silently ran the experimental DCP chunked path without the opt-in and without the warning, violating fail-closed. Add resolve_effective_chunked_prefill (new lightweight header scheduler/chunked_prefill_policy.h) and share it across the scheduler batch-mode resolution, the DCP compatibility gate, and the startup warning so the effective chunked semantics cannot drift. Correct the gate error and warning to name the multi_slo_and_prio implicit enable and the full rollback conditions. Extend dcp_compat and scheduler tests with the multi-SLO and no-op experimental-flag cases. Co-Authored-By: Claude <noreply@anthropic.com>

The DCP experimental gate now treats multi_slo_and_prio as implicitly enabling chunked prefill, but several downstream consumers still read the raw enable_chunked_prefill flag, so a gate-approved multi_slo_and_prio + raw-false config could still fail or misbehave at service level: - llm_engine.cpp linear-attention prefix-cache precondition CHECK would reject the config even though the gate allowed it (Qwen3.5 GDN + prefix on the experimental path); - llm_master.cpp prompt admission would clamp long prompts as if chunked prefill were off; - continuous_scheduler.cpp activation metric mislabeled the batch. Route all three through resolve_effective_chunked_prefill (scheduler reads the already-resolved batch_mode_), add master.cpp a direct include of the helper header, add a truth-table test for the predicate, and drop the duplicate MoE model-type test. Co-Authored-By: Claude <noreply@anthropic.com>

Remove the MoE model-type rejection from the DCP first-version gate so Qwen3.5 MoE can run with decode_context_parallel_size>1. The GQA topology checks (head divisibility) still apply and are independent of the expert layers. MoE on this path is not bitwise-equivalent to decode_context_parallel_size=1: the DCP communication/merge floating point order can produce a measurable per-step logprob difference on the MoE-sensitive path (observed ~0.044 at one teacher-forced step on 35B). This is accepted for now as a functional-first capability; the kernel path fix is future work. The validator became an unconditional no-op once the branch was gone, so drop the function, its call site, and its tests entirely rather than leave a dead shim. Co-Authored-By: Claude <noreply@anthropic.com>

MoE + decode_context_parallel_size>1 is allowed but not bitwise-equivalent to dcp=1: the DCP decode attention kernel path (FIA) differs from dcp=1 (batch_decode), and MoE expert routing amplifies it into a per-step logprob delta on the order of 1e-2 (roughly an order of magnitude larger than the chunked-prefill BF16 quantization diff). Log a one-time startup warning so the non-equivalence is visible to operators. The unified kernel path fix is future work. Co-Authored-By: Claude <noreply@anthropic.com>

Eager schedule overlap is compatible with DCP: per-step DCP metadata is recomputed each forward and DCP collectives allocate fresh buffers per call, and the scheduler/worker FIFO + compute-stream ordering prevents step i+1 from clobbering step i state. Remove the first-version schedule-overlap rejection in validate_dcp_first_version_options (graph path stays blocked independently by the attention paged_attention_tiling_data CHECK). On-card: overlap dcp2 output is token-exact vs non-overlap dcp2 (5 dense cases, tp4/dcp2). Co-Authored-By: Claude <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds Ascend NPU support for Qwen3.5 Decode Context Parallel (DCP), enabling KV/context sharding within existing TP groups for decode (and gated experimental chunked prefill), including ACLGraph capture/replay integration and scheduler safety handling.

Changes:

Introduces DCP topology/config plumbing end-to-end (flags/config/options, process groups, spawn protocol, master validation/gating).
Implements DCP attention paths (decode + experimental chunked prefill) using FIA partials + FP32 online-softmax merge, plus stable ACLGraph task-update support.
Adds targeted unit/integration tests and build wiring for new DCP utilities and probes.

Reviewed changes

Copilot reviewed 62 out of 62 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
xllm/xllm.cpp	Wires DCP-related ParallelConfig fields into runtime Options creation.
xllm/core/scheduler/continuous_scheduler.h	Adds decode_context_parallel_size option and documents mixed-batch restriction under DCP.
xllm/core/scheduler/continuous_scheduler.cpp	Resolves effective chunked prefill consistently and disables mixed batching when DCP is enabled.
xllm/core/scheduler/chunked_prefill_policy.h	Adds shared predicate for “effective chunked prefill” (raw flag + multi_slo_and_prio).
xllm/core/runtime/worker_impl.h	Declares DCP cache-slot remap helper.
xllm/core/runtime/worker_impl.cpp	Remaps cache slots for DCP ownership and updates kv_slot_layout for NPU DCP local physical layout.
xllm/core/runtime/options.h	Adds decode_context_parallel_size to runtime Options.
xllm/core/runtime/forward_params.h	Adds KvSlotLayout::NPU_DCP_LOCAL_PHYSICAL enum value.
xllm/core/runtime/acl_graph_persistent_param.h	Adds persistent DCP-local block-table buffers and update hook for decode graph replay.
xllm/core/runtime/acl_graph_persistent_param.cpp	Allocates and updates stable DCP-local block tables; threads them into graph params during decode.
xllm/core/runtime/acl_graph_executor_impl.h	Exposes graph key helper for tests.
xllm/core/runtime/acl_graph_executor_impl.cpp	Adds FIA task-update support for DCP replay; adds replay barrier; refines graph key bucketing for non-DCP decode.
xllm/core/platform/npu/acl_graph_task_update_context.h	Extends task-update context to record DCP FIA graph tasks with stable tensors/workspace.
xllm/core/layers/npu_torch/qwen3_next_attention.cpp	Passes DCP parameters/process group into NPU attention module construction.
xllm/core/layers/npu_torch/dcp_attention_utils.h	Declares DCP helper utilities (local KV lens, normalization, chunked validation, merge).
xllm/core/layers/npu_torch/dcp_attention_utils.cpp	Implements DCP helper utilities, including online-softmax merge for partials.
xllm/core/layers/npu_torch/CMakeLists.txt	Adds dcp_attention_utils library and links it into NPU torch layers.
xllm/core/layers/npu_torch/attention.h	Extends AttentionImpl constructor for DCP parameters; adds DCP forward helpers.
xllm/core/layers/npu_torch/attention.cpp	Implements DCP decode and chunked-prefill attention (FIA partials + allgather + merge) and ACLGraph capture support.
xllm/core/layers/common/attention_metadata.h	Adds DCP-local block table and ACLGraph task-update context to AttentionMetadata.
xllm/core/layers/common/attention_metadata_builder.cpp	Populates new DCP/graph-task-update fields into AttentionMetadata under NPU.
xllm/core/kernels/npu/npu_ops_api.h	Adds FIA out-variant and workspace-size query APIs for stable-address ACLGraph capture.
xllm/core/kernels/npu/npu_fused_infer_attention.cpp	Refactors FIA invocation to support out-variant, optional caller workspace, and workspace sizing.
xllm/core/kernels/npu/aclnn/pytorch_npu_helper.hpp	Adds macros to query workspace size and run ops with caller-provided workspace buffer.
xllm/core/framework/parallel_state/parallel_state.h	Adds DCP group ranks computation, cache-slot remap helpers, and local block-table selection API.
xllm/core/framework/parallel_state/parallel_state.cpp	Implements DCP ranks computation, cache-slot ownership/remap, and local block-table selection.
xllm/core/framework/parallel_state/parallel_args.h	Adds dcp_size and derived dcp_size_effective/dcp_rank helpers.
xllm/core/framework/parallel_state/collective_communicator.h	Adds DCP process group ownership in communicator.
xllm/core/framework/parallel_state/collective_communicator.cpp	Creates DCP process groups within TP groups and wires them into ParallelArgs.
xllm/core/framework/model/model_input_params.h	Adds DCP-local block tables to GraphInput and device transfer path.
xllm/core/framework/config/parallel_config.h	Adds DCP size and experimental chunked-prefill flag to ParallelConfig schema.
xllm/core/framework/config/parallel_config.cpp	Registers/loads/dumps new ParallelConfig fields and defines corresponding flags.
xllm/core/distributed_runtime/worker_server.cpp	Passes decode_context_parallel_size through spawn worker argv.
xllm/core/distributed_runtime/vlm_master.cpp	Threads decode_context_parallel_size into VLM scheduler options.
xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server.h	Extends SpawnWorkerServer ctor to accept decode_context_parallel_size.
xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server.cpp	Wires decode_context_parallel_size into runner options and parallel args.
xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server_process.cpp	Parses/validates decode_context_parallel_size from argv and logs topology.
xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_protocol.h	Extends spawn protocol argument count and adds DCP argument index.
xllm/core/distributed_runtime/rec_master.cpp	Threads decode_context_parallel_size into Rec scheduler options.
xllm/core/distributed_runtime/master.cpp	Adds DCP model/config validation, experimental gating + warnings, and propagates DCP size into config/options.
xllm/core/distributed_runtime/llm_master.cpp	Threads decode_context_parallel_size into scheduler options and uses effective chunked-prefill resolution.
xllm/core/distributed_runtime/llm_engine.cpp	Uses effective chunked-prefill resolution for prefix-cache requirements and clarifies error guidance.
xllm/core/distributed_runtime/dcp_compat.h	Adds centralized “first version” DCP compatibility/gating rules (chunked, disagg PD, speculative, etc.).
xllm/core/common/options.h	Adds decode_context_parallel_size and experimental DCP chunked-prefill option to public Options.
xllm/core/common/options.cpp	Extends Options::to_string() with decode_context_parallel_size.
xllm/core/common/global_flags.h	Declares decode_context_parallel_size flag.
xllm/api_service/completion_service_impl.cpp	Refactors usage proto filling into shared helper.
tests/core/scheduler/continuous_scheduler_test.cpp	Adds tests for DCP mixed-batch closure and effective chunked-prefill behavior.
tests/core/runtime/acl_graph_executor_test.cpp	Adds tests for task-update context, graph key bucketing, and stable DCP-local block tables in persistent params.
tests/core/layers/npu_torch/dcp_attention_utils_test.cpp	Adds CPU-side tests for DCP merge/validation helpers.
tests/core/layers/npu_torch/dcp_attention_test.cpp	Adds NPU tests validating DCP attention behaviors (normalization, slicing, chunked first chunk fast path).
tests/core/layers/npu_torch/CMakeLists.txt	Adds build targets for DCP attention tests/util tests.
tests/core/kernels/npu/fia_decode_lse_probe_test.cpp	Adds NPU probe tests for FIA decode output/LSE correctness and DCP-relevant metadata behavior.
tests/core/kernels/npu/CMakeLists.txt	Adds build target for FIA decode LSE probe test.
tests/core/framework/parallel_state/cp_group_ranks_test.cpp	Extends parallel_state tests for DCP ranks, slot remap, local block-table selection, and prefix-cache ownership contracts.
tests/core/framework/parallel_state/CMakeLists.txt	Links additional deps needed by extended parallel_state tests under NPU.
tests/core/framework/config/config_json_test.cpp	Extends config json tests for new DCP flags/options and flag-guard coverage.
tests/core/distributed_runtime/spawn_worker_protocol_test.cpp	Adds assertion that DCP argument is appended at protocol tail.
tests/core/distributed_runtime/dcp_compat_test.cpp	Adds unit tests for DCP compatibility gating logic.
tests/core/distributed_runtime/CMakeLists.txt	Adds dcp_compat_test build target.
tests/core/common/options_test.cpp	Extends options default/explicit-value tests to include decode_context_parallel_size.
cmake/cc_test.cmake	Adds NO_NPU_RUNTIME option to run CPU-only tests without NPU runtime setup.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+  if (engine_type != EngineType::LLM && engine_type != EngineType::SSM) {
+    return "decode context parallelism supports only LLM text generation";
+  }


+  const torch::Tensor pos = positions.to(torch::kCPU).to(torch::kLong);
+  const torch::Tensor slot = slots.to(torch::kCPU).to(torch::kLong);
+  const torch::Tensor owner =
+      torch::floor_divide(pos, interleave_size) % dcp_size;
+  const torch::Tensor mask = (owner == dcp_rank) & (slot >= 0);
+  return torch::where(mask, slot, torch::full_like(slot, -1, slot.options()));


Preserve DCP configuration while integrating layerwise KV-cache sharding, and fail closed when both modes are requested.

…rectness

yingxudeng · 2026-08-20T10:50:38Z

  cmake_parse_arguments(
    CC_TEST # prefix
-    "" # options
+    "NO_NPU_RUNTIME" # options


为什么突然加这个

单测遗留的忘了删了，对项目无意义，我回退一下

Revert cmake/cc_test.cmake to upstream. NO_NPU_RUNTIME was added only to build the DCP attention-math unit test as CPU-only, an unnecessary change to shared test infrastructure. Build that test as a standard NPU test (matching deepseek_v4_eplb_load_utils_test) instead; cmake/cc_test.cmake is now byte-identical to upstream.

- Drop dead decode_context_parallel_size pass-through for VLM/SSM/REC masters. Only LLM reaches DCP (startup gate allows LLM/SSM and dcp_compat rejects SSM), so those builder calls were unreachable. - Remove the test-only production scalar compute_dcp_cache_slot; move the owner/slot oracle into cp_group_ranks_test as expected_dcp_cache_slot (cross-checks the production tensor remap_dcp_cache_slots path) and drop the three self-tests that only exercised the helper. - Revert the completion_service_impl usage change (deferred to a separate API PR); the file is byte-identical to upstream again. - Link the production :dcp_attention_utils target from its unit test instead of recompiling the source.

The per-replay current-stream synchronize (added to prevent the DCP FIA graph cross-replay task/event overlap that surfaced as ACL_ERROR_RT_MODEL_ EXECUTE 507011) was gated on hybrid linear attention, so it also blocked the ordinary dcp=1 GDN conv graph replay hot path, for which no such fault has been observed. Gate it on the graph having captured FIA tasks (has_fia_graph_tasks) so only DCP decode graphs pay the barrier; dcp=1 GDN graphs keep their non-blocking replay.

…rectness # Conflicts: # xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_protocol.h # xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server.cpp # xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server.h # xllm/core/distributed_runtime/spawn_worker_server/spawn_worker_server_process.cpp # xllm/core/distributed_runtime/worker_server.cpp

Enguikong and others added 14 commits August 5, 2026 11:11

feat: enable prefix cache for DCP

185f39d

feat: support DCP-aware chunked prefill

def4450

merge: reconcile fork DCP branch history

15e5217

fix: close Phase 4 DCP validation gaps

1533da6

fix: fail closed DCP chunked prefill

c80271c

Enguikong changed the title ~~feat: add decode context parallel (DCP) for Qwen3.5 GQA dense.~~ feat: Decode Context Parallel for Qwen3.5 GQA (dense+MoE) — chunked prefill + prefix cache Aug 12, 2026

Enguikong mentioned this pull request Aug 12, 2026

bugfix: clean NPU process shutdown (bypass process-static destructors). #2209

Open

Enguikong changed the title ~~feat: Decode Context Parallel for Qwen3.5 GQA (dense+MoE) — chunked prefill + prefix cache~~ feat: Decode Context Parallel for Qwen3.5 GQA (dense+MoE) — chunked prefill + prefix cache + schedule overlap Aug 13, 2026

feat: support decode context parallelism with ACL graph

3844bd5

Enguikong changed the title ~~feat: Decode Context Parallel for Qwen3.5 GQA (dense+MoE) — chunked prefill + prefix cache + schedule overlap~~ feat: support Qwen3.5 Decode Context Parallel on Ascend NPU Aug 17, 2026

Merge upstream/main into feat/qwen35-dcp-correctness

ab6003e

Enguikong marked this pull request as ready for review August 20, 2026 06:15

Copilot AI lite review requested due to automatic review settings August 20, 2026 06:15

Enguikong requested review from DongheJin, DragonFive, JimHsiung, Kang-Meng, liutongxuan, yingxudeng and zhang-minchao as code owners August 20, 2026 06:15

Enguikong requested review from liujinguang0125, ustcfy, xiao-yu-chen and yinjiawei01 as code owners August 20, 2026 06:15

Copilot started reviewing on behalf of Enguikong August 20, 2026 06:16 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Enguikong added 3 commits August 20, 2026 15:21

style: format DCP attention declaration

ffdb8bd

Merge upstream/main into feat/qwen35-dcp-correctness

4f64c0a

Preserve DCP configuration while integrating layerwise KV-cache sharding, and fail closed when both modes are requested.

Merge remote-tracking branch 'upstream/main' into feat/qwen35-dcp-cor…

e5efe19

…rectness

yingxudeng reviewed Aug 20, 2026

View reviewed changes

yingxudeng changed the title ~~feat: support Qwen3.5 Decode Context Parallel on Ascend NPU~~ feat: support Qwen3.5 Decode Context Parallel on Ascend NPU. Aug 20, 2026

Enguikong added 8 commits August 20, 2026 20:29

Merge upstream/main into feat/qwen35-dcp-correctness

805d9c4

fix: align GLM5.2 expert loading with lazy allocation

7ee62fc

fix: register Python bridge modules after interpreter init

e201280

ci: select NPU with visible device mask

93830fd

Navigation Menu

Uh oh!

feat: support Qwen3.5 Decode Context Parallel on Ascend NPU. - #2125

feat: support Qwen3.5 Decode Context Parallel on Ascend NPU.#2125
Enguikong wants to merge 28 commits into
xLLM-AI:mainfrom
Enguikong:feat/qwen35-dcp-correctness

Enguikong commented Aug 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

yingxudeng Aug 20, 2026

Uh oh!

Enguikong Aug 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Enguikong commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Qwen3.5 Decode Context Parallel（DCP）Ascend NPU 适配

概要

背景与动机

设计

DCP 拓扑与 KV ownership

Attention 路径

Chunked prefill 安全门

Mixed-batch 安全

Schedule overlap

ACLGraph

正确性契约

验证结果

离线验证

上卡功能验证

大模型 EvalScope

性能观察

已知限制

不包含的范围

Reviewer 关注点

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

yingxudeng Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

Enguikong Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Enguikong commented Aug 5, 2026 •

edited

Loading