-
Notifications
You must be signed in to change notification settings - Fork 6.4k
Add update expected / unexpected keys api to DiffusersQuantizer #12471
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Disty0
wants to merge
4
commits into
huggingface:main
Choose a base branch
from
Disty0:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+42
−7
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -233,7 +233,7 @@ def load_model_dict_into_meta( | |
empty_state_dict = model.state_dict() | ||
|
||
for param_name, param in state_dict.items(): | ||
if param_name not in empty_state_dict: | ||
if unexpected_keys is not None and param_name in unexpected_keys: | ||
continue | ||
|
||
set_module_kwargs = {} | ||
|
@@ -260,10 +260,16 @@ def load_model_dict_into_meta( | |
# For compatibility with PyTorch load_state_dict which converts state dict dtype to existing dtype in model, and which | ||
# uses `param.copy_(input_param)` that preserves the contiguity of the parameter in the model. | ||
# Reference: https://github.com/pytorch/pytorch/blob/db79ceb110f6646523019a59bbd7b838f43d4a86/torch/nn/modules/module.py#L2040C29-L2040C29 | ||
old_param = model | ||
splits = param_name.split(".") | ||
for split in splits: | ||
old_param = getattr(old_param, split) | ||
if param_name in empty_state_dict: | ||
old_param = model | ||
splits = param_name.split(".") | ||
for split in splits: | ||
old_param = getattr(old_param, split) | ||
else: | ||
# hf_quantizer can add parameters that doesn't exist yet in the model and the empty_state_dict | ||
# they will be created in create_quantized_param and hf_quantizer should handle the loading of these parameters | ||
# these parameters will be in the loaded_state_dict from the model file instead when loading a pre_quantized model | ||
old_param = None | ||
|
||
if not isinstance(old_param, (torch.nn.Parameter, torch.Tensor)): | ||
old_param = None | ||
|
@@ -279,7 +285,9 @@ def load_model_dict_into_meta( | |
|
||
# bnb params are flattened. | ||
# gguf quants have a different shape based on the type of quantization applied | ||
if empty_state_dict[param_name].shape != param.shape: | ||
# current parameter might not be in the empty_state_dict if the hf_quantizer needs to create it in create_quantized_param | ||
# pass the to be created parameters to create_quantized_param instead | ||
if param_name in empty_state_dict and empty_state_dict[param_name].shape != param.shape: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just add a small comment for that as we will probably refactor the loading at some point to match what we have in transformers There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added: 745b041 |
||
if ( | ||
is_quantized | ||
and hf_quantizer.pre_quantized | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah that's better, actually in transformers we rely on unexpected keys