Fix division-by-zero NaN in shift_terminal with a single denoising step - #14415
Open
mayuriphad wants to merge 1 commit into
Open
Fix division-by-zero NaN in shift_terminal with a single denoising step#14415mayuriphad wants to merge 1 commit into
mayuriphad wants to merge 1 commit into
Conversation
stretch_shift_to_terminal() rescales sigmas so the schedule ends at config.shift_terminal, using scale_factor = one_minus_z[-1] / (1 - shift_terminal). With num_inference_steps=1 the only sigma is always 1.0, so one_minus_z[-1] is 0, scale_factor is 0, and the division produces NaN. That NaN sigma then breaks index_for_timestep() with an IndexError during scheduler.step(). There is nothing to stretch with a single step, so skip the call when len(sigmas) <= 1. Applied the same guard to the three schedulers that support shift_terminal: FlowMatchEulerDiscreteScheduler, FlowMatchLCMScheduler, and UniPCMultistepScheduler (flow-sigmas path). Fixes huggingface#14411 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes a division-by-zero NaN in stretch_shift_to_terminal() when num_inference_steps=1 and shift_terminal is enabled, by skipping the terminal-stretch rescaling when there鈥檚 only a single sigma value. This prevents NaN sigmas from propagating into index_for_timestep() / scheduler.step().
Changes:
- Guard
stretch_shift_to_terminal()application withlen(sigmas) > 1in all schedulers that supportshift_terminal. - Add/adjust inline rationale comments explaining why the single-step case is skipped (FlowMatch schedulers).
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py | Skip terminal stretch when only one sigma to prevent NaN/div-by-zero in single-step inference. |
| src/diffusers/schedulers/scheduling_flow_match_lcm.py | Apply the same single-step guard for shift_terminal stretching (LCM variant). |
| src/diffusers/schedulers/scheduling_unipc_multistep.py | Apply the single-step guard on the UniPC flow-sigmas path to avoid the same NaN edge case. |
Suppressed comments (1)
src/diffusers/schedulers/scheduling_flow_match_euler_discrete.py:355
- The explanatory comment here is slightly too absolute:
set_timestepsallows customsigmas/timesteps, so withnum_inference_steps=1the single sigma is not necessarily1.0. The division-by-zero rationale applies to the default schedule (and typical usage), so it would be clearer to qualify the statement to avoid misleading future readers.
# 3. If required, stretch the sigmas schedule to terminate at the configured `shift_terminal` value. This is
# skipped when there is only a single step, since there is nothing to stretch and the terminal rescaling
# otherwise divides by zero (the single sigma is always 1.0, i.e. `one_minus_z[-1]` is always 0).
馃挕 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+353
to
357
| # 3. If required, stretch the sigmas schedule to terminate at the configured `shift_terminal` value. This is | ||
| # skipped when there is only a single step, since there is nothing to stretch and the terminal rescaling | ||
| # otherwise divides by zero (the single sigma is always 1.0, i.e. `one_minus_z[-1]` is always 0). | ||
| if self.config.shift_terminal and len(sigmas) > 1: | ||
| sigmas = self.stretch_shift_to_terminal(sigmas) |
Comment on lines
+362
to
+364
| # 3. If required, stretch the sigmas schedule to terminate at the configured `shift_terminal` value. This is | ||
| # skipped when there is only a single step, since there is nothing to stretch and the terminal rescaling | ||
| # otherwise divides by zero (the single sigma is always 1.0, i.e. `one_minus_z[-1]` is always 0). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #14411
Root cause
stretch_shift_to_terminal()rescales the sigma schedule so it terminates atconfig.shift_terminal, computingscale_factor = one_minus_z[-1] / (1 - shift_terminal). Whennum_inference_steps=1, the single sigma is always1.0, soone_minus_z[-1] == 0, makingscale_factor == 0and the division produceNaN. ThatNaNsigma then crashesindex_for_timestep()with anIndexErrorinsidescheduler.step().Fix
There is nothing to stretch with only one step, so skip the call to
stretch_shift_to_terminal()whenlen(sigmas) <= 1. The sameif self.config.shift_terminal:pattern (guarding a call tostretch_shift_to_terminal) exists in three schedulers that supportshift_terminal, so the guard is applied consistently to all three:FlowMatchEulerDiscreteSchedulerFlowMatchLCMSchedulerUniPCMultistepScheduler(flow-sigmas path)None of the touched lines are inside
# Copied fromblocks, so nomake fix-copiesfollow-up is needed.Verification
FlowMatchEulerDiscreteScheduler.set_timesteps(num_inference_steps=1, ...)withshift_terminal=0.1against the patched code: sigmas are[1., 0.], no NaN.num_inference_steps=4is unaffected and still correctly terminates atshift_terminal=0.1([1.0, 0.7, 0.4, 0.1, 0.0]).