Skip to content

SM<90 default attention path (flash_hub) currently crashes at first forward — ecosystem regression; document/auto-fallback --disable_flash_attention #140

Description

@smcshahid

Summary. On GPUs below SM90, infer_helios.py:258 and app.py:34 select diffusers'
flash_hub backend (per PR #28's hardware detection). Since ~June 2026 that path crashes at
the first attention call: the kernels-community/flash-attn2 hub artifact renamed its private
wrappers, and diffusers' registry (stable 0.39.0 and current main) still references the old
_wrapped_flash_attn_* names — tracked upstream as huggingface/diffusers#14012 (open; partial
fix PR #14015 covers only varlen). Net effect: the documented recipes crash on
4090/A6000/A40/L40S-class hardware
unless the user knows about --disable_flash_attention.

Environment. Helios @ 8f2a2fa; torch 2.10.0+cu128; diffusers 0.39.0 (also
reproduced on git-main); kernels 0.13.0 (repo pin; also 0.16.0); RTX 4090 (SM89), driver 580.

Reproduction. Run any official SM<90 recipe, e.g.
bash scripts/inference/helios-distilled_t2v.sh on a 4090 → after model load, first
denoising step raises
AttributeError: Kernel module 'flash_attn2_…' does not define attribute path 'flash_attn_interface._wrapped_flash_attn_forward'.

Expected. Generation proceeds (as it did when PR #28 landed in March).

Suggested fixes (either/both, minimal):

  1. README/FAQ note: on SM<90 add --disable_flash_attention until diffusers#14012 is fixed
    (PyTorch SDPA path works; we measured ~7.3 FPS vs 10.7 FPS compiled-FA3 on H100 for scale).
  2. In infer_helios.py, wrap the set_attention_backend("flash_hub") call in try/except with
    a logged fallback to SDPA — cheap resilience against exactly this class of hub/registry
    drift (the FA3 branch already has a try/except precedent at infer_helios.py:252-255).

Impact. All consumer/workstation GPUs (the community PR #28 explicitly targeted 4090)
crash on default settings; the failure message doesn't mention the existing escape hatch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions