Feature Request: Auto-detect encrypted volumes and disable mmap to prevent system freezes
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
## Feature Idea
When running ComfyUI on a VeraCrypt-encrypted volume, loading large models (7GB+ safetensors checkpoints) causes severe system freezing or complete unresponsiveness.
**Root cause analysis:**
VeraCrypt's driver opens host files with `FILE_NO_INTERMEDIATE_BUFFERING`, completely bypassing the Windows file system cache. Meanwhile, ComfyUI defaults to memory-mapped file I/O (mmap) when loading safetensors via `safetensors.safe_open()`. The combination creates a catastrophic I/O pattern:
1. Every mmap page fault triggers a physical disk read through VeraCrypt's synchronous encryption pipeline (256KB fragments, XTS-AES decrypt on each)
2. VeraCrypt's I/O pipeline threads run at `LOW_REALTIME_PRIORITY`, starving normal-priority threads (Python/UI)
3. With no OS cache, repeated accesses to the same tensor data cause repeated physical reads
A 7GB model creates ~28,000 fragmented reads through the encryption layer. The system appears frozen because VeraCrypt's real-time threads consume all available CPU for AES operations while the UI/compute threads can't schedule.
**The existing `--disable-mmap` flag fixes this** (line 174 in cli_args.py, used at line 137 in comfy/utils.py). When mmap is disabled, safetensors are loaded eagerly as CPU tensors (sequential large reads), which work efficiently with VeraCrypt's read-ahead cache.
However, `--disable-mmap` requires user manual discovery. The fix should be automatic.
**Suggested implementation:**
1. **Auto-detect encrypted volumes**: At startup, check if the model directory resides on a volume opened with `FILE_NO_INTERMEDIATE_BUFFERING` or other encryption indicators.
2. **Fall back gracefully**: When an encrypted or no-cache volume is detected, automatically treat `--disable-mmap` as enabled, or print a prominent warning suggesting the user enable it.
3. **Simpler alternative**: Change the default of `--disable-mmap` to `True` on Windows, or at minimum print a startup warning about potential issues with encrypted volumes.
## Existing Solutions
The `--disable-mmap` CLI flag already exists. The `comfy/utils.py` line 137-138 shows:
```python
if DISABLE_MMAP:
tensor = tensor.to(device=device, copy=True)
```
There is also a TODO comment: `"# TODO: Not sure if this is the best way to bypass the mmap issues"`.
## Other
I have analyzed both VeraCrypt's source code (veracrypt/VeraCrypt, specifically `src/Driver/EncryptedIoQueue.c` and `src/Driver/Ntvol.c`) and ComfyUI's model loading path (`comfy/utils.py`, `comfy/memory_management.py`, `comfy/model_management.py`) to arrive at this root cause analysis.
@comfyanonymous
Contributor guide
Assessment
This issue has not been assessed yet.