Skip to content

Conversation

@kevincheng2
Copy link
Collaborator

@kevincheng2 kevincheng2 commented Nov 27, 2025

Motivation

💡 If this PR is a Cherry Pick, the PR title needs to follow the format by adding the [Cherry-Pick] label at the very beginning and appending the original PR ID at the end. For example, [Cherry-Pick][CI] Add check trigger and logic(#5191)

💡 如若此PR是Cherry Pick,PR标题需遵循格式,在最开始加上[Cherry-Pick]标签,以及最后面加上原PR ID,例如[Cherry-Pick][CI] Add check trigger and logic(#5191)

修复多模perfix cache 和 cudagraph 同时开启时,cuda error 700问题

cuda error 问题是由于多模调度方案修改,在高并发下会插入多条请求,导致显存使用比较极限,该问题在 #5181 已经修复

Modifications

Usage or Command

默认打开多模prefix caching 和 cudagraph

Accuracy Tests

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

@paddle-bot
Copy link

paddle-bot bot commented Nov 27, 2025

Thanks for your contribution!

@yuanlehome yuanlehome requested a review from Copilot November 27, 2025 06:26
Copilot finished reviewing on behalf of yuanlehome November 27, 2025 06:28
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

这个PR修复了多模态(multi-modal)模型同时启用prefix cache和CUDAGraph时出现的cuda error 700问题。通过移除之前强制禁用prefix caching的限制代码,允许这两个特性可以同时工作。

Key Changes

  • 移除了在fastdeploy/config.py中当同时启用多模态和CUDAGraph时强制禁用prefix caching的逻辑

@codecov-commenter
Copy link

codecov-commenter commented Nov 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (develop@db936ab). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #5266   +/-   ##
==========================================
  Coverage           ?   59.08%           
==========================================
  Files              ?      326           
  Lines              ?    40321           
  Branches           ?     6112           
==========================================
  Hits               ?    23823           
  Misses             ?    14653           
  Partials           ?     1845           
Flag Coverage Δ
GPU 59.08% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants