Skip to content

Fix GGUF reload race causing heap corruption on Apple Silicon #2021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ArFeRR wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix GGUF reload race causing heap corruption on Apple Silicon #2021

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions nodes_sampler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,8 +126,19 @@ def process(self, model, image_embeds, shift, steps, cfg, seed, scheduler, rifle
block_swap_args=block_swap_args, compile_args=model["compile_args"])

if gguf_reader is not None: #handle GGUF
load_weights(transformer, patcher.model["sd"], base_dtype=dtype, transformer_load_device=device, patcher=patcher, gguf=True,
reader=gguf_reader, block_swap_args=block_swap_args, compile_args=model["compile_args"])
# Skip GGUF reload after the first call to avoid heap corruption on
# Apple Silicon (MPS): repeatedly re-reading the multi-GB GGUF blob
# races MPS async frees against CPU allocations in unified memory
# and triggers BUG IN CLIENT OF LIBMALLOC on the 2nd render onward.
# Set WANVIDEO_DISABLE_GGUF_RELOAD_GUARD=1 to restore the original
# behaviour (needed for mid-process LoRA hot-swap).
_gguf_reload_guard = (
os.environ.get("WANVIDEO_DISABLE_GGUF_RELOAD_GUARD", "0") != "1"
)
if not _gguf_reload_guard or not getattr(transformer, "_gguf_weights_loaded", False):
load_weights(transformer, patcher.model["sd"], base_dtype=dtype, transformer_load_device=device, patcher=patcher, gguf=True,
reader=gguf_reader, block_swap_args=block_swap_args, compile_args=model["compile_args"])
transformer._gguf_weights_loaded = True
set_lora_params_gguf(transformer, patcher.patches)
transformer.patched_linear = True
elif len(patcher.patches) != 0: #handle patched linear layers (unmerged loras, fp8 scaled)
Expand Down