Repository navigation
[Common] Extract Functions to Get and Validate Group Sizes in Weights Compression - #4237
Conversation
|
|
||
| mode: CompressWeightsMode = CompressWeightsMode.INT8_ASYM | ||
| group_size: int = -1 | ||
| group_size: int | None = None |
There was a problem hiding this comment.
group_size is annotated as int | None, even though __post_init__ resolves it to an int.
But Mypy still treats it as optional elsewhere, for example:
channel_size // group_size a possible None operand`
mypy isn't turned on for this files yet, but refactoring like this will only complicate its adoption across the whole project.
Looking at the implementation, now i see that passing an internal config to the external API seems like a bad idea. We won't be able to change internal details later without breaking/changing the external API.
There was a problem hiding this comment.
That is a good point. We can still create a thin wrapper on top of WeightCompressionConfig to expose externally. it might be overcomplicating though.
| DEFAULT_GROUP_SIZE_FOR_NON_INT8_MODES = 128 | ||
|
|
||
|
|
||
| def get_default_group_size(mode: CompressWeightsMode, group_size: int) -> int: |
There was a problem hiding this comment.
- If it's named get_default_group_size, semantically it should only take mode and use.
The problem is that the function does more than its name implies, which is likely why the incorrect type annotation was added.
group_size or get_default_group_size(model)
# or
if group_size is None:
group_size = get_default_group_size(model)It's ok if you keep as it, but fix group_size: int | None) and rename function to
resolve_group_size or get_group_size_or_default
And idea to the next PRs, move some function in seprate files, algorithms.py now contains more 1000 lines
There was a problem hiding this comment.
Yes, good point. I will move the check for group_size is None from the function to call site.
since it is better if the function does 1 small nuclear thing
I agree with moving some functions and will do it in a follow up PR.

Changes
Rework the weights compression algorithm to make reusable getter for default and fixed group sizes and also for validating group sizes for different modes.
Reason for changes
To be reused by scoped config feature
Related tickets
CVS-180366
#4171
Tests