Skip to content

Conversation

Disty0
Copy link
Contributor

@Disty0 Disty0 commented Oct 11, 2025

What does this PR do?

Adds update_expected_keys and update_unexpected_keys APIs to DiffusersQuantizer.
Makes load_model_dict_into_meta compatible with updated unexpected / expected keys added in DiffusersQuantizer.

Fixes #12470

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Core library:

@Disty0
Copy link
Contributor Author

Disty0 commented Oct 11, 2025

Here is a use case of this PR with SDNQ:

pip install git+https://github.com/Disty0/sdnq
import torch
import diffusers
from sdnq import SDNQConfig # import sdnq to register it into diffusers and transformers

pipe = diffusers.FluxPipeline.from_pretrained("Disty0/FLUX.1-dev-SDNQ-uint4-svd-r32", torch_dtype=torch.bfloat16)

if (
    hasattr(pipe.text_encoder_2.encoder.block[0].layer[0].SelfAttention.k, "scale")
    and pipe.text_encoder_2.encoder.block[0].layer[0].SelfAttention.k.scale.device.type != "meta"
    and hasattr(pipe.transformer.single_transformer_blocks[0].attn.to_k, "scale")
    and pipe.transformer.single_transformer_blocks[0].attn.to_k.scale.device.type != "meta"
):
    print("SDNQ model loaded succesfully")
else:
    print("SDNQ model failed to load")
    exit()

pipe.enable_model_cpu_offload()
prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save("flux-dev-sdnq-uint4-svd-r32.png")

(This result is generated with an Intel ARC A770 with FP64 Rope downcasted to FP32 to make it work on Alchemist as it doesn't support FP64.)
Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add update_unexpected_keys and update_expected_keys APIs to DiffusersQuantizer

1 participant