Windows access violation when loading large (>10GB) safetensors CLIP in long-running process
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
## Bug Report: Windows access violation when loading large (>10GB) safetensors CLIP/Text Encoder in long-running process
### Environment
- **OS**: Windows 11, RTX 5080 16GB, 48GB RAM, pagefile 128GB
- **ComfyUI**: 0.30.0 (Windows portable, python_embeded 3.13 / also reproduced on py312)
- **torch**: 2.13.0+cu130 (also reproduced on 2.11.0+cu128 stable and 2.12.0.dev nightly)
- **Model**: MiniMax H3 Qwen3VL text encoder (qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors, 15.6GB, NVFP4 quantized)
### Bug
ComfyUI process crashes with `Windows fatal exception: access violation` when executing a prompt that loads a >10GB safetensors CLIP/text encoder via `CLIPLoader` node.
Stack (most recent call first):
```
torch/storage.py line 471 in __getitem__ (UntypedStorage.__getitem__ -> super().__getitem__)
comfy/utils.py line 136 in load_torch_file (f.get_tensor(k) inside safetensors.safe_open)
comfy/sd.py line 1454 in load_clip
comfy/nodes.py line 1015 in load_clip (CLIPLoader node)
```
### Key observation
The crash **only** occurs in the ComfyUI long-running process. The exact same code path (`safetensors.safe_open(...).get_tensor()` on the same file, same device, same torch) succeeds in standalone Python processes — tested: serial, concurrent threads, asyncio, CUDA-init-then-thread, with embedding_directory, with aimdo imported, all pass. Crash is deterministic in ComfyUI process, regardless of `--disable-mmap`, `--lowvram`/`--novram`, torch version, or Python version.
This suggests a memory-layout/allocator interaction between the ComfyUI long-running process and safetensors mmap on Windows for files >10GB.
### Workaround (verified)
Using `safetensors.torch.load_file` with `backend="pread"` (non-mmap read) instead of `safe_open` avoids the crash entirely:
```python
# in comfy/utils.py load_torch_file, when DISABLE_MMAP or COMFYUI_LOAD_FILE_FULL=1:
sd = safetensors.torch.load_file(ckpt, device=device.type, backend="pread")
```
Verified: with this change, MiniMax H3 (40GB model set) generates video successfully on the above hardware. Without it, 100% crash rate.
### Suggestion
Extend `--disable-mmap` to also bypass the mmap inside `safetensors.safe_open` (currently it only adds `tensor.to(copy=True)` after `get_tensor`, which does not prevent the mmap-based read that crashes). Using `safetensors.torch.load_file(backend="pread")` as the non-mmap path would cover both this Windows crash and the network-storage use case from #2288.
Contributor guide
Research direction
Start in comfy/utils.py at load_torch_file and trace how DISABLE_MMAP and COMFYUI_LOAD_FILE_FULL affect safetensors.safe_open and get_tensor. Reproduce with the reported large Windows CLIP/text encoder, then verify the non-mmap path avoids the access violation while preserving normal model loading.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100