Skip to content

Basic convrot int4 support. #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
comfyanonymous merged 9 commits into from
Jul 9, 2026
Merged

Basic convrot int4 support. #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d2a742f
Basic convrot int4 support.
comfyanonymous Jul 8, 2026
cc511a2
Overflow fix on int4.
comfyanonymous Jul 8, 2026
5d5e849
Merge remote-tracking branch 'origin/main' into dev/int4
comfyanonymous Jul 8, 2026
4e2518f
Missing tests.
comfyanonymous Jul 8, 2026
769f324
bring up int4 to same speed as int8 on blackwell
comfyanonymous Jul 9, 2026
ac3d717
Support forcing the convrot int4 layers to run the matrix mult in int8.
comfyanonymous Jul 9, 2026
93d19e1
Fix issues with large activations.
comfyanonymous Jul 9, 2026
8cd18b9
Merge remote-tracking branch 'origin/main' into dev/int4
comfyanonymous Jul 9, 2026
314f4bc
Update function used to calculate shared mem usage.
comfyanonymous Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions comfy_kitchen/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@

# Import registry and exceptions
from .registry import registry
from .tensor.convrot_w4a4 import (
convrot_w4a4_linear,
dequantize_convrot_w4a4_weight,
quantize_convrot_w4a4_weight,
)

__version__ = "0.1.0"

Expand All @@ -31,13 +36,16 @@
"quantize_mxfp8",
"dequantize_mxfp8",
"quantize_svdquant_w4a4",
"quantize_convrot_w4a4_weight",
"quantize_int8_rowwise",
"quantize_int8_tensorwise",
"dequantize_int8_simple",
# Fused matmul
"scaled_mm_nvfp4",
"scaled_mm_mxfp8",
"scaled_mm_svdquant_w4a4",
"convrot_w4a4_linear",
"dequantize_convrot_w4a4_weight",
"gemv_awq_w4a16",
"int8_linear",
# Positional encoding
Expand Down
1 change: 1 addition & 0 deletions comfy_kitchen/backends/cuda/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ set(CUDA_SOURCES
ops/cublas_gemm_int8.cu
ops/cutlass_gemm_int8.cu
ops/int8_linear.cu
ops/convrot_w4a4.cu
ops/apply_rope.cu
ops/quantize_svdquant_w4a4.cu
ops/scaled_mm_svdquant_w4a4.cu
Expand Down
666 changes: 652 additions & 14 deletions comfy_kitchen/backends/cuda/__init__.py
View file
Open in desktop

Large diffs are not rendered by default.

Loading
Loading