Dynamic VRAM + Symlinked Model causes model reload on every prompt after workflow/model switch (v0.23)

# Bug Report: Dynamic VRAM causes model reload on every prompt after workflow/model switch

## Regarding Custom Nodes
I was unable to test with custom nodes disabled, as the workflow requires them to function. However, based on the logs, the trigger point correlates exactly with a model dtype change (fp16 → bf16+nvfp4 mixed), not with any custom node behavior. The slowdown occurs at the ComfyUI core level (Dynamic VRAM model management), and I believe this is a core bug rather than a custom node issue.

## Environment
- **ComfyUI Version:** v0.23.0-10-gbd7da053a (Released 2026-06-02)
- **OS:** Windows 11
- **Python:** 3.13.11
- **PyTorch:** 2.12.0+cu130
- **GPU:** NVIDIA GeForce RTX 5070 Ti (16GB VRAM)
- **RAM:** 64GB
- **Driver:** 610.47

## Expected Behavior
After the first load, models (ZImageTEModel_, Lumina2) should remain staged in Dynamic VRAM and not reload on every prompt. Repeated runs with only a seed change should complete quickly (~23-28 seconds in my case).

## Actual Behavior
After switching workflows or loading a different model mid-session, models begin reloading on every subsequent prompt. Once triggered, the slowdown persists for the rest of the session without recovery.

- Normal runs: ~23-28 seconds
- After trigger: ~120-190 seconds per run

## Steps to Reproduce
1. Start ComfyUI with Dynamic VRAM enabled (no `--disable-dynamic-vram`)
2. Run a workflow several times — runs are fast after first load
3. Switch to a different workflow or load a model with different dtype/quantization (e.g. switching between fp16 and bf16/nvfp4 mixed models)
4. Return to original workflow
5. Observe that `Requested to load` now appears on every prompt and execution time increases 5-8x

## Key Observation
The trigger appears to be a model dtype/quantization change mid-session. In the log below, normal fast runs are interrupted when a workflow switch causes:
```
[INFO] model weight dtype torch.bfloat16, manual cast: None
[INFO] model_type FLOW
[INFO] Requested to load ZImageTEModel_ ← starts appearing every run from this point
```
Before this point, `Requested to load` does NOT appear on repeated runs.

Note: This issue does NOT occur when using `--disable-dynamic-vram`. However, per the warning message, that flag is a temporary workaround and will be removed.

---

## Log

### Section 1: Normal behavior (fast runs after first load)

```
[INFO] got prompt
[INFO] Using pytorch attention in VAE
[INFO] Using pytorch attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load ZImageTEModel_
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[INFO] model weight dtype torch.float16, manual cast: None
[INFO] model_type FLOW
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. 0 patches attached. Force pre-loaded 145 weights: 383 KB.
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: mxfp8, nvfp4, float8_e5m2, float8_e4m3fn
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged. 0 patches attached. Force pre-loaded 265 weights: 1046 KB.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged. 0 patches attached. Force pre-loaded 265 weights: 1046 KB.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00, 2.44s/it]
[INFO] Requested to load AutoencodingEngine
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:11<00:00, 1.87s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Prompt executed in 167.80 seconds ← first run (includes initial load)

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00, 2.58s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:10<00:00, 1.82s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Prompt executed in 23.16 seconds ← fast

[INFO] got prompt
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged. ← no "Requested to load"
[INFO] 0 models unloaded.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] 0 models unloaded.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:12<00:00, 4.17s/it]
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [00:10<00:00, 1.82s/it]
[INFO] 0 models unloaded.
[INFO] Prompt executed in 27.95 seconds ← fast, no "Requested to load" at all
```

### Section 2: Trigger point — workflow/model switch causes dtype reinitialization

```
[INFO] got prompt
[INFO] model weight dtype torch.bfloat16, manual cast: None ← model reinit triggered by workflow switch
[INFO] model_type FLOW
[INFO] Requested to load ZImageTEModel_ ← "Requested to load" returns
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: mxfp8, nvfp4, float8_e5m2, float8_e4m3fn
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 3/3 [00:07<00:00, 2.62s/it]
[INFO] Requested to load AutoencodingEngine
[INFO] 0 models unloaded.
[INFO] Model AutoencodingEngine prepared for dynamic VRAM loading. 159MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 11738MB Staged. 180 patches attached.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [01:52<00:00, 18.78s/it] ← suddenly 18s per step vs 1.8s before
[INFO] Prompt executed in 186.56 seconds ← 8x slower
```

### Section 3: Slowdown persists — no recovery without restart

```
[INFO] got prompt
[INFO] Requested to load ZImageTEModel_ ← persists every run
[INFO] Model ZImageTEModel_ prepared for dynamic VRAM loading. 7671MB Staged.
[INFO] Requested to load Lumina2
[INFO] Model Lumina2 prepared for dynamic VRAM loading. 8504MB Staged.
(RES4LYF) rk_type: res_2m
100%|████████████████████| 6/6 [01:52<00:00, 18.69s/it]
[INFO] Prompt executed in 183.31 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 176.45 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 124.70 seconds

[INFO] got prompt
[INFO] Requested to load ZImageTEModel_
...
[INFO] Prompt executed in 130.12 seconds
```

---

## Additional Notes
- Issue does NOT reproduce on a fresh ComfyUI start for the first few runs
- Issue triggers reliably after switching workflows that involve different model dtypes (fp16 → bf16+nvfp4 mixed)
- Once triggered, only a full ComfyUI restart restores normal speed
- `--disable-dynamic-vram` prevents the issue but is documented as a temporary flag to be removed
- This behavior was NOT present in v0.22 with the same workflows and hardware

Workflow : [Z+Z V3.zip](https://github.com/user-attachments/files/28587475/Z%2BZ.V3.zip)