[anzr299]

Changes

Add the GroupedMatMul operation for data-free weight compression

Reason for changes

Support the GroupedMatMul operation in torch backend

Related tickets

CVS-191922

Tests

test_get_reduction_axes

Test examples - success

Weight compression - success

[Copilot]

Copilot review overview

🟡 Changes recommended

Critical failures remain in the example output path and dynamic GroupedMatMul offsets handling.

Review effort: Lite
Findings: 2 · 1

Open (3)

• Hard-coded output path may not exist · New
• GroupedMatMul activation port incorrectly includes offsets · New
• GroupedMatMul incorrectly advertised as supported by FX backend · New

What changed in this PR

Adds Torch GroupedMatMul support for data-free weight compression and a tiny Qwen3 MoE example.

Changes:

• Registers GroupedMatMul and adds grouped-weight axis handling.
• Adds transpose-aware graph traversal and activation handling.
• Adds tests, documentation, dependencies, and an example.

File	Summary
tests/​torch/​test_model_graph_manager.py	Adds reduction-axis coverage; backend layout coverage remains incomplete.
src/​nncf/​torch/​node_utils.py	Updates grouped matmul activation-axis handling.
src/​nncf/​torch/​model_graph_manager.py	Adds grouped-weight traversal; permute handling needs alignment with transpose.
src/​nncf/​torch/​graph/​operator_metatypes.py	Registers GroupedMatMul; FX backend compatibility requires handling.
src/​nncf/​quantization/​algorithms/​weight_compression/​torch_backend.py	Adds grouped reduction-axis logic; dynamic offsets handling requires correction.
examples/​llm_compression/​torch/​tiny_random_qwen3_moe/​requirements.txt	Adds example dependencies.
examples/​llm_compression/​torch/​tiny_random_qwen3_moe/​README.md	Documents the example setup and usage.
examples/​llm_compression/​torch/​tiny_random_qwen3_moe/​main.py	Adds the compression example; its hard-coded graph output path is invalid for users.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

Moderate issues remain in the example, grouped-matmul coverage, metatype integrations, and transpose handling.

Review effort: Lite
Findings: 2

Open (2)

• Add end-to-end Torch compression coverage for GroupedMatMul · New
• Recognize all transpose aliases in constant weight tracing · New

Resolved since last review (3)

• GroupedMatMul activation port incorrectly includes offsets
• Hard-coded output path may not exist
• GroupedMatMul incorrectly advertised as supported by FX backend

Previously missed (2)

In code that hasn't changed since last review

Request a mapping from apply_chat_template

examples/​llm_compression/​torch/​tiny_random_qwen3_moe/​main.py:52

apply_chat_template returns the token tensor by default when return_dict is not set, even when return_tensors="pt" is requested. Consequently batch_feature is a Tensor here and batch_feature["input_ids"] raises before the example can generate any output. Request a mapping explicitly.

Avoid misquantizing GroupedMatMul in shared matmul paths

src/​nncf/​torch/​graph/​operator_metatypes.py:1054

Adding this metatype to the shared MATMUL_METATYPES list also makes it a supported matmul for the regular Torch MinMax quantization path (src/nncf/quantization/algorithms/min_max/torch_backend.py:66). That path still calls get_weight_channel_axes, whose fallback returns axis 0 for GroupedMatMul; an untransposed grouped weight has [num_groups, C_IN, C_OUT] layout and will therefore be quantized along the wrong dimension (and the generic path has no grouped-mm offsets handling). Keep this operation out of the shared list and expose a weight-compression-specific list, or add equivalent grouped-mm handling to every consumer of the list.

[andreyanufr]

@anzr299
Could you add to

nncf/src/nncf/torch/quantization/layers.py

Line 990 in 524037a

out_features, in_features = lspec.orig_weight_shape

support of 3d weight from group_mm in this PR?

[anzr299]

@anzr299 Could you add to

nncf/src/nncf/torch/quantization/layers.py

Line 990 in 524037a

out_features, in_features = lspec.orig_weight_shape

support of 3d weight from group_mm in this PR?

Done

[andreyanufr]

I ran model training using GroupedMatMul. Everything works except for the import into OpenVINO.

[anzr299]

I ran model training using GroupedMatMul. Everything works except for the import into OpenVINO.

Is the import into openvino issue related to the PR?

[andreyanufr]

I ran model training using GroupedMatMul. Everything works except for the import into OpenVINO.

Is the import into openvino issue related to the PR?

No, it is a problem with model patching and compressed weigths.