feat(wan video): Introduce transition_video for seamless action-transfer concatenation, compatible with context mode and Wan Animate loop

[wuwukaka]

Motivation

Currently, concatenating video batches for action transfer heavily relies on the start_ref_image parameter. However, under the current logic, using this parameter forces the generation into the looping (Wan Animate) mode and is architecturally incompatible with the context (context windowing) mode. In long video generation, relying solely on the looping mode leads to error accumulation, resulting in noticeable visual degradation in subsequent frames.

This PR introduces the transition_video parameter, allowing users to input a short video clip as a hard conditioning guide. This approach removes the restriction that video concatenation must be strictly bound to the looping mode, making it compatible with both context mode and Wan Animate loop mode. This grants users the option to utilize the context mode during action-transfer concatenation to maintain higher visual stability.

Usage

1. Node Connection: In the WanVideo Wrapper node, connect the prepared transition video to the newly added transition_video input.
2. Mode Configuration: This interface is agnostic to the generation mode. Users can either connect context parameters via a standalone Context node to prevent visual degradation, or retain the Wan Animate chunked settings; the concatenation guidance will function correctly in both scenarios.

Implementation Details by File

1. nodes.py (Wrapper Node & Preprocessing)
Responsible for robust video preprocessing and strictly aligning the timing of control signals.

• Parameter Injection & Canvas Expansion: Added the transition_video input. When provided, the total num_frames is forced to expand by 32 frames (corresponding to 8 latent frames).
• Control Signal Padding: Prepends padding to spatial/conditioning control signals (pose_images, face_images, bg_images, mask). Extracts the 0-th frame of each signal, duplicates it 32 times using .repeat(32, ...), and prepends it to the original sequence, ensuring the original control actions are accurately shifted to the 33rd frame.
• Video Preprocessing & Precise Frame Alignment:
  • Frame Correction: Sets the target frame count to 32. If input frames b > 32, uses torch.linspace to generate uniform indices for downsampling; if b < 32, calculates the ceiling repeat_factor to duplicate the video tensor, then truncates it using [:32] slicing, ensuring the output is strictly 32 frames.
  • Spatial & Value Adjustment: Reshapes and resizes the aligned video to the target dimensions, adjusts dimensions to [C, T, H, W], extracts the first 3 RGB channels, and normalizes the values to [-1, 1] via * 2 - 1.
• VAE Encoding & Hard Mask: Encodes the processed 32-frame video into an 8-frame transition_latent via VAE; constructs an all-1.0 tensor of length 8 as transition_mask_values, instructing the sampler to execute a 100% hard replacement.
• Conflict Resolution: If both transition_video and start_ref_image are provided, logs a warning and prioritizes the transition_video pipeline.

2. nodes_sampler.py (Sampler)
Responsible for precisely slicing and replacing the starting tensors of the target latent sequence during the initialization phase.

• Context Windowing Mode Adaptation:

  • Mask Channel Construction: Before each denoising step, extracts the first 4 channels from the 36-channel image_cond as window_msk.
  • Precise Tensor Replacement: When processing the first window (c[0] == 0), slices partial_img_emb. Skips the first 4 mask channels and directly replaces the target latent section (channels 4:20, frames 1:1+transition_len) with transition_latent.
  • Mask Application: Assigns transition_mask_values to the corresponding timesteps of window_msk, and overwrites the first 4 channels of the input condition image_cond_in before passing it to the model prediction.

• Wan Animate (Looping) Mode Adaptation:

  • Trigger Condition: Appends the has_transition flag to the logic evaluating the reconstruction of temporal_ref_latents.
  • Image Concatenation Reconstruction: Independently extracts current_ref_images and bg_image_slice, concatenates them using torch.cat, and sends them to VAE encoding.
  • Condition Injection: When processing the first chunk (start == 0), retains the original target latent sequence after transition_len, and hard-replaces the first 8 frames with transition_latent.

Impact

• Backward Compatibility: When no transition video is connected, all newly added preprocessing and replacement logic is skipped, leaving the original workflow and performance completely unaffected.

[kslz]

Does this support I2V and T2V? Is the input frame length configurable, or is it hardcoded to 32 frames?

[kslz]

something wrong

[wuwukaka]

something wrong

Thanks for pointing this out. You were absolutely right—there was a minor issue where the transition padding falsely triggered the built-in loop. I've just pushed a fix for this.