Skip to content

inference(ltx-video): offload path for legacy 13B BF16 (or deprecate it) #599

Description

@jdilley

Legacy LTX-Video 13B BF16 requires full transformer residency and errors out when VRAM is short: "MOLD_OFFLOAD is not implemented for this legacy transformer yet" (crates/mold-inference/src/ltx_video/pipeline.rs:398, as of 1503696). Every other large family routes through the shared AdaptiveResidencyPlan planner.

Scope: wire the legacy LTX-Video transformer into the adaptive offload/streaming path (or the block-streaming runtime pattern used by FLUX/SD3/Z-Image), so 13B dev/distilled run on 24 GB-class cards instead of erroring.

Worth a quick priority check first: if LTX-2 fully supersedes LTX-Video 0.9.8 for users, deprecating 13B BF16 may be the better resolution than building offload for it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestinferenceInference engine changes

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions