Comfy-Org / Comfy-Org/ComfyUI

Feature Request: Auto-detect encrypted volumes and disable mmap to prevent system freezes

Open
#14,006 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

## Feature Idea

When running ComfyUI on a VeraCrypt-encrypted volume, loading large models (7GB+ safetensors checkpoints) causes severe system freezing or complete unresponsiveness.

**Root cause analysis:**

VeraCrypt's driver opens host files with `FILE_NO_INTERMEDIATE_BUFFERING`, completely bypassing the Windows file system cache. Meanwhile, ComfyUI defaults to memory-mapped file I/O (mmap) when loading safetensors via `safetensors.safe_open()`. The combination creates a catastrophic I/O pattern:

1. Every mmap page fault triggers a physical disk read through VeraCrypt's synchronous encryption pipeline (256KB fragments, XTS-AES decrypt on each)
2. VeraCrypt's I/O pipeline threads run at `LOW_REALTIME_PRIORITY`, starving normal-priority threads (Python/UI)
3. With no OS cache, repeated accesses to the same tensor data cause repeated physical reads

A 7GB model creates ~28,000 fragmented reads through the encryption layer. The system appears frozen because VeraCrypt's real-time threads consume all available CPU for AES operations while the UI/compute threads can't schedule.

**The existing `--disable-mmap` flag fixes this** (line 174 in cli_args.py, used at line 137 in comfy/utils.py). When mmap is disabled, safetensors are loaded eagerly as CPU tensors (sequential large reads), which work efficiently with VeraCrypt's read-ahead cache.

However, `--disable-mmap` requires user manual discovery. The fix should be automatic.

**Suggested implementation:**

1. **Auto-detect encrypted volumes**: At startup, check if the model directory resides on a volume opened with `FILE_NO_INTERMEDIATE_BUFFERING` or other encryption indicators.

2. **Fall back gracefully**: When an encrypted or no-cache volume is detected, automatically treat `--disable-mmap` as enabled, or print a prominent warning suggesting the user enable it.

3. **Simpler alternative**: Change the default of `--disable-mmap` to `True` on Windows, or at minimum print a startup warning about potential issues with encrypted volumes.

## Existing Solutions

The `--disable-mmap` CLI flag already exists. The `comfy/utils.py` line 137-138 shows:
```python
if DISABLE_MMAP:
tensor = tensor.to(device=device, copy=True)
```

There is also a TODO comment: `"# TODO: Not sure if this is the best way to bypass the mmap issues"`.

## Other

I have analyzed both VeraCrypt's source code (veracrypt/VeraCrypt, specifically `src/Driver/EncryptedIoQueue.c` and `src/Driver/Ntvol.c`) and ComfyUI's model loading path (`comfy/utils.py`, `comfy/memory_management.py`, `comfy/model_management.py`) to arrive at this root cause analysis.

@comfyanonymous

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.