Intermittent SEGV in archive_model_dtypes / named_buffers when loading a checkpoint shortly after a dynamic-VRAM model load (timing race, reproduces on current master)

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

The second model load should either succeed or fail with a normal Python
exception (as partially_load/archive_model_dtypes do on other error
paths) — never a process-level segfault.

### Actual Behavior

Description

Running two different models back-to-back — e.g. a Wan2.2 video model
(UNETLoader, dynamic-VRAM-loaded) followed shortly after by an unrelated
FLUX checkpoint (CheckpointLoaderSimple) — intermittently segfaults the
whole ComfyUI process. Not every attempt crashes; it appears to be a race
tied to how soon the second load starts after the first model's dynamic-VRAM
staging. In our environment (mediad, a small orchestration layer that frees
VRAM cooperatively before submitting the next job) shortening the delay
before submitting the next prompt made the crash noticeably more
frequent, which points at a timing-sensitive race rather than a
deterministic bug in one code path.

The crash is always inside PyTorch's own named_buffers/named_parameters
(_named_members), called from comfy/model_management.py's
archive_model_dtypes, itself called from comfy/sd.py's
load_state_dict_guess_config → load_checkpoint_guess_config →
nodes.py's CheckpointLoaderSimple.load_checkpoint. We also saw the same
partially_load/_load_list/named_parameters path segfault from a
VAEDecode node during the same kind of transition, so the crash isn't
specific to CheckpointLoaderSimple — it's the dynamic-VRAM / async
weight-offload machinery generally. Neither captured stack trace shows any
custom-node frame — both go directly from ComfyUI's own execution.py into
comfy/sd.py/comfy/model_management.py into PyTorch.

Actual behavior — crash 1 (commit 175e854, 2026-05-27)

Fatal Python error: Segmentation fault
Stack (most recent call first):
File "torch/nn/modules/module.py", line 2775 in named_buffers
File "comfy/model_management.py", line 964 in archive_model_dtypes
File "comfy/sd.py", line 246 in __init__
File "comfy/sd.py", line 1892 in load_state_dict_guess_config
File "comfy/sd.py", line 1740 in load_checkpoint_guess_config
File "nodes.py", line 607 in load_checkpoint
File "execution.py", line 298 in process_inputs
File "execution.py", line 310 in _async_map_node_over_list
File "execution.py", line 336 in get_output_data
File "execution.py", line 536 in execute
File "execution.py", line 774 in execute_async
...
File "main.py", line 327 in prompt_worker

This exact trace was captured with ComfyUI-Lora-Manager completely
removed from custom_nodes/ (renamed out, ComfyUI restarted, confirmed
via startup log that it did not load) — the stack goes directly from
execute() to execute_async() with no custom-node wrapping at all. An
earlier occurrence of the identical trace (same line numbers, same call
chain) did show ComfyUI-Lora-Manager/py/metadata_collector/metadata_hook.py
wrapping the same two execution.py frames, which is what led us to
suspect that node first — the side-by-side pair (present vs. fully absent,
identical line numbers) rules that out as a cause, though see "Custom node
sensitivity" below for the more rigorous all-nodes-disabled test.

Actual behavior — crash 2 (commit ffbecfff, 2026-07-07, i.e. current master)

Fatal Python error: Segmentation fault
Stack (most recent call first):
File "torch/nn/modules/module.py", line 2663 in _named_members
File "torch/nn/modules/module.py", line 2721 in named_parameters
File "comfy/model_management.py", line 984 in archive_model_dtypes
File "comfy/sd.py", line 251 in __init__
File "comfy/sd.py", line 1945 in load_state_dict_guess_config
File "comfy/sd.py", line 1793 in load_checkpoint_guess_config
File "nodes.py", line 629 in load_checkpoint
...

Same call chain, same crash site (archive_model_dtypes → __init__ →
load_state_dict_guess_config → load_checkpoint_guess_config →
load_checkpoint), only the line numbers shifted (~20 lines, consistent
with unrelated changes across the six-week gap) and the specific PyTorch
entry point varies (named_buffers vs. named_parameters/
_named_members — both are the same family of "iterate this module's
tensors" call). Again, no custom-node frame anywhere in the trace.

### Steps to Reproduce

Minimal repro

Load and run any workflow using a large UNETLoader-based model (we used
Wan2.2 TI2V-5B, wan2.2_ti2v_5B_fp16.safetensors, ~9.3GB) so it's
resident in VRAM via dynamic VRAM loading.
Within a few seconds of that job completing, submit a second, unrelated
workflow that loads a different checkpoint via CheckpointLoaderSimple
(we used FLUX.1-schnell fp8, flux1-schnell-fp8.safetensors, ~16GB) —
ideally while VRAM is still under some pressure from the first model, so
the loader takes the partial/dynamic-VRAM path rather than a full clean
load.
Repeat several times. It doesn't reproduce every time (we saw roughly
1-in-4 to 1-in-2 depending on how quickly step 2 followed step 1) —
consistent with a race rather than a deterministic bug.

We don't have a single-file API-format workflow pair isolated yet (happy to
produce one if useful) — this was found via an orchestration layer that
issues the two prompts over HTTP, not the ComfyUI UI directly. Attached:
wan22_5b_t2v.json and flux_schnell_t2i.json, both API-format, both using
only core node types.

Attached: comfyui-segv-repro-workflows.zip — both workflows API-format, core node types only.

[comfyui-segv-repro-workflows.zip](https://github.com/user-attachments/files/29799381/comfyui-segv-repro-workflows.zip)

### Debug Logs

```powershell
Note on logs: the segfault itself occurred under our production systemd
service; the two faulthandler stack traces captured from its journal are
included in full in the Actual Behavior section above (complete journal
excerpts available on request). The attached comfyui-issue-logs.txt is the
full log of the isolated --disable-all-custom-nodes reproduction attempt
(process start → end of test), including the hang addendum. Its annotated
header, for context:

ComfyUI SEGV repro — all-custom-nodes-disabled test
=====================================================

Context: this log documents an attempt to reproduce the
archive_model_dtypes / named_buffers SEGV (see comfyui-upstream-issue-draft.md)
on a manual, isolated ComfyUI instance launched with --disable-all-custom-nodes
and WITHOUT the two mitigation flags (--disable-async-offload,
--disable-dynamic-vram) normally applied in production. Same models dir,
spare port (8001), real service stopped for the duration.

RESULT: no segfault was captured. 7 real, non-cached wan(t2v)->flux(t2i)
transitions ran to completion clean: 0/7.

Two methodology notes, both disclosed rather than hidden:

1. Lines ~110-330 below (the first transition, seed=0, the checked-in
default in both workflow JSONs) is a genuine, real transition: clean.

2. The next block ("Prompt executed in 0.00 seconds" x10) are NOT real
repro attempts. The first repro-loop script resubmitted the identical
workflow JSON (same seed, same every input) five more times.
ComfyUI's node-level output cache recognized the unchanged input graph
and returned the cached result instantly without reloading any model —
so these five "attempts" never exercised the code path under test.
This was caught and corrected before drawing any conclusion from it.

3. Everything after that is the corrected test: 6 further transitions,
each with a freshly randomized KSampler seed on both the Wan and the
Flux workflow (forcing a real cache-miss and a real model reload every
time), submitted back-to-back with no artificial delay. All 6
completed clean. Combined with the one genuine attempt in (1), that's
7/7 real transitions clean, 0 segfaults, 0 hangs, under
--disable-all-custom-nodes.

A different failure mode (NOT the SEGV under test) was observed on an
earlier launch of this same isolated instance: a Wan render hung
indefinitely at step 16/30 of the KSampler loop, GPU utilization idle,
process still alive (no crash, no exception, no exit). That log section
is appended as a separate addendum at the end of this file, since it's
the same dynamic-VRAM/offload subsystem and may be relevant to upstream
even though it's a distinct symptom from the segfault.
```

### Other

Environment

GPU: NVIDIA GeForce RTX 5090, 32607MiB VRAM, driver 610.43.02, CUDA 13.0
OS: Linux (CachyOS, Arch-based), Python 3.11.14
PyTorch: 2.11.0+cu130
ComfyUI: reproduced on two commits, six weeks apart —

175e85466a05ee00ba3f9e1ed9898190e3e0d17a (2026-05-27)
ffbecfffb953914f5b4bd8f61d810ff2300631de (2026-07-07, "Fix crash when
using UNetSelfAttentionMultiply") — i.e. current master at time of
writing

Launch flags (on the older commit, before this was diagnosed): none
special — dynamic VRAM + async weight offloading enabled by default (as
documented, "Enabled by default on Nvidia")
Custom nodes present when it reproduces: several (ComfyUI-Manager,
rgthree-comfy, comfyui-easy-use, Impact-Pack/Subpack, kjnodes,
videohelpersuite, ComfyUI-Lora-Manager, others) — see "Custom node
sensitivity" below: it also reproduces with ComfyUI-Lora-Manager, the
node present in every early trace, completely removed from
custom_nodes/, and it did NOT reproduce in 7/7 real attempts with
every custom node disabled. We do not believe any custom node is the
root cause, but we can't rule out a custom node acting as a timing
perturbation that makes the underlying race more likely to fire.

Custom node sensitivity

We do not believe this is a custom-node bug, but we tested as rigorously
as we could and want to report that honestly rather than overclaim.

We removed ComfyUI-Lora-Manager (the node present in every early
trace) entirely and it reproduced with an identical signature (see
crash 1 above) — that specific node is ruled out.
We then launched a fully isolated instance with ComfyUI's
--disable-all-custom-nodes flag (zero custom nodes loaded at all,
confirmed via startup log) and WITHOUT our two mitigation flags, and
ran the same Wan→FLUX transition repro 7 times, each with a freshly
randomized sampler seed on both workflows (to force a real model
reload every time rather than hit ComfyUI's node-output cache). All 7
completed clean — it did not reproduce.

Custom Node Testing checkbox: NOT ticked. Honestly, we can't tick it
in good conscience. 7 clean runs is consistent with "this is genuinely
independent of custom nodes and we didn't get unlucky" (our observed
crash rate elsewhere has been roughly 1-in-4 to 1-in-2, so 0/7 is on the
low-probability side but not implausible), but it's also consistent with
some custom node acting as a timing perturbation — e.g. adding enough
extra work/scheduling jitter between the two loads — that makes an
underlying, node-independent race more likely to fire, without being the
cause of the race itself. We don't have a way to distinguish those two
explanations from the data we have. We're reporting both the "no custom
nodes, still didn't reproduce" result and the earlier "no
Lora-Manager, still reproduced with an identical trace" result and
leaving the interpretation open.

What we tried

Updating ComfyUI-Lora-Manager to the latest release (v1.1.6) — crash
persisted.
Removing ComfyUI-Lora-Manager entirely — crash persisted with an
identical trace (see above). This rules it out.
Updating ComfyUI core from the 2026-05-27 pin to current master
(2026-07-07) — crash persisted with the same signature.
Launching with --disable-all-custom-nodes — did not reproduce in 7/7
real (non-cached, randomized-seed) attempts. See "Custom node
sensitivity" above.
Workaround that resolved it for us: launching with
--disable-async-offload --disable-dynamic-vram (switches to
NORMAL_VRAM / estimate-based loading). 0 crashes in 5 back-to-back
repro attempts after adding both flags together; --disable-async-offload
alone was insufficient (2/4 clean — partially_load still appeared in
the trace), so the bug appears to live in the broader dynamic-VRAM
partial-load mechanism, not only the async-offload streaming
optimization. Cost: ~4.3s slower per model transition in our environment.

Happy to provide more detail, run a debug build, or produce an isolated
API-format workflow pair if that's more useful than the HTTP-driven repro
above.

Addendum: a second, distinct failure mode in the same subsystem

While preparing the all-custom-nodes-disabled test above, on one launch of
the same isolated instance (same flags: --disable-all-custom-nodes, no
mitigation flags) a Wan2.2 render hung indefinitely at step 16/30 of
the KSampler loop — progress output stopped advancing, GPU utilization
dropped to idle, and the process itself stayed alive (no exception, no
exit, no log output at all past that point). This is a different symptom
from the segfault above (hang vs. crash) but the same subsystem
(dynamic-VRAM model loading during a Wan2.2 render) under the same
launch conditions, so we're including it in case it's a related or
downstream effect of the same race. We did not investigate this further
since it's outside the scope of the segfault repro; the process was
killed manually after confirming it was genuinely stuck (VRAM held,
nvidia-smi utilization at 0%, no progress for well past the normal
render time). Full log from process start through the point progress stopped is included

[comfyui-issue-logs.txt](https://github.com/user-attachments/files/29799489/comfyui-issue-logs.txt)

in the Debug Logs section above (addendum section at the end of the log).

Start by reproducing the Wan→FLUX transition with the attached API-format workflows, then read comfy/model_management.py around archive_model_dtypes and the partially_load path. Trace the calls through comfy/sd.py's load_state_dict_guess_config and load_checkpoint_guess_config, plus nodes.py's CheckpointLoaderSimple and the VAEDecode path. Done means repeated dynamic-VRAM transitions either complete or raise normal Python exceptions, never a process-level segfault.

Written by the indexing model from the issue text.