Skip to content

refactor: consolidate speculative-validate profiling gating and fitting. - #2274

Open
weizhehuang0827 wants to merge 1 commit into
xLLM-AI:mainfrom
weizhehuang0827:spec-profile-refactor
Open

refactor: consolidate speculative-validate profiling gating and fitting.#2274
weizhehuang0827 wants to merge 1 commit into
xLLM-AI:mainfrom
weizhehuang0827:spec-profile-refactor

Conversation

@weizhehuang0827

Copy link
Copy Markdown
Collaborator

Description

Pure refactor of the speculative validate-time profiling in ProfileManager,
no behavior change to the adaptive speculative controller.

  • Hoist the adaptive gate to the call site. The decision "only profile the
    validate-time predictor when adaptive speculative decode is enabled" now
    lives in should_profile_speculative_validate() and is checked at the
    ProfileManager ctor where profiling is orchestrated, instead of being
    buried inside profile_speculative_validate_time(). The function keeps a
    defensive self-guard so a direct call cannot profile a config the controller
    would never consume.
  • Move the fit into TimePredictor. The least-squares fit + MAE/MAPE +
    coefficient sanitization moves into TimePredictor::fit_for_speculative_ validate, matching the existing fit_for_decode / fit_for_prefill.
    ProfileManager::train_speculative_validate_time_predictor now only maps the
    fitted coefficients into the registry struct and keeps the
    broadcast-first-then-commit publish path (unchanged).
  • Strengthen check_coefficients_non_neg to also clamp non-finite
    (NaN/Inf) coefficients, not just negatives. A rank-deficient QR solve or a
    NaN latency sample can otherwise slip through a negative-only check
    (NaN < 0 is false). This matches the sanitizer the speculative path
    previously did inline and now benefits the prefill/decode predictors too.

Test

  • Builds clean (xllm + all_tests).
  • Runtime check on DeepSeek-V4-Flash, dp=2, adaptive on: the validate-time
    predictor still fits and the fit now logs from
    TimePredictor::fit_for_speculative_validate
    (Fitted speculative validate equation: ...), confirming the delegation
    path. The adaptive controller consumes the predictor exactly as before.

- Hoist the adaptive gate out of profile_speculative_validate_time() to the
  ProfileManager ctor call site via should_profile_speculative_validate(), so
  the "only profile when adaptive speculative decode is enabled" intent is
  visible where profiling is orchestrated (the function keeps a defensive
  self-guard).
- Move the least-squares fit + MAE/MAPE + coefficient sanitization into
  TimePredictor::fit_for_speculative_validate, matching fit_for_decode /
  fit_for_prefill. ProfileManager::train_speculative_validate_time_predictor
  now only maps the fitted coefficients into the registry struct and keeps the
  broadcast-first-then-commit publish path.
- Strengthen TimePredictor::check_coefficients_non_neg to also clamp
  non-finite (NaN/Inf) coefficients, not just negatives; a rank-deficient QR
  solve or NaN latency sample could otherwise pass a negative-only check.

Pure refactor: same design matrix, same publish contract, no behavior change
to the adaptive controller.
@weizhehuang0827 weizhehuang0827 changed the title refactor: consolidate speculative-validate profiling gating and fitting refactor: consolidate speculative-validate profiling gating and fitting. Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants