Skip to content

Make API inference_steps defaults model-aware to prevent noisy XL-SFT generations - #1223

Draft
ChuxiJ with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-noise-in-sft-model
Draft

Make API inference_steps defaults model-aware to prevent noisy XL-SFT generations#1223
ChuxiJ with Copilot wants to merge 3 commits into
mainfrom
copilot/fix-noise-in-sft-model

Conversation

Copilot AI commented May 27, 2026

Copy link
Copy Markdown
Contributor

API generations were defaulting to inference_steps=8 regardless of model, which is appropriate for turbo but under-sampled for SFT variants and can surface as noisy output. This aligns API behavior with model-specific defaults already used in UI paths.

  • Model-aware defaulting in request build path

    • Updated acestep/api/http/release_task_request_builder.py to derive default inference_steps from requested model name when the field is omitted:
      • turbo8
      • sft50
      • explicit non-turbo/non-sft models → 32
      • no model provided → legacy default 8 (preserved for backward compatibility)
    • Added token-boundary model detection (turbo/sft) to avoid partial-string false positives.
  • API docs updated for effective defaults

    • Updated docs/en/API.md and .claude/skills/acestep-docs/api/API.md to document model-aware default behavior for inference_steps.
  • Focused regression/unit coverage

    • Expanded acestep/api/http/release_task_request_builder_test.py to cover:
      • SFT/turbo/base/unspecified model defaults
      • token boundary behavior and precedence (turbo over sft when both appear)
      • case-insensitive model handling
      • parser compatibility for seed fallback behavior used by existing parser test fakes
def _default_inference_steps_for_model(model_name: Optional[str]) -> int:
    normalized = (model_name or "").strip().lower()
    if not normalized:
        return 8  # legacy default when model is omitted
    if _has_model_token("turbo", normalized):
        return 8
    if _has_model_token("sft", normalized):
        return 50
    return 32

Copilot AI linked an issue May 27, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Fix noise issue in API with XL-SFT model Make API inference_steps defaults model-aware to prevent noisy XL-SFT generations May 27, 2026
Copilot AI requested a review from ChuxiJ May 27, 2026 23:13
Copilot finished work on behalf of ChuxiJ May 27, 2026 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Getting Noise In API With XL-SFT MODEL

2 participants