Comfy-Org / Comfy-Org/ComfyUI

memory_required() estimate misses masked attention fallback and per-model attention overrides

Open
#16,021 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

**What we found**
Two gaps in the flash-attention memory estimate added by #15586, both flagged by Codex on that PR:
1. Masked attention falls back to plain SDPA in `attention_flash()`, not real flash attention. `memory_required()` still returns the cheap estimate for it.
2. The `ModelAttentionBackend` node can override a model's attention backend at runtime. `memory_required()` can't see that override, so it can return the cheap estimate on a model that isn't actually running flash attention.

**What we want**
`memory_required()` should never return the cheap estimate when the real attention path isn't flash.

Follow-up to #15586, tracking the deferred items from that review.

Contributor guide

Open the contributing guide

Research direction

Start by locating memory_required(), attention_flash(), and the ModelAttentionBackend node, then inspect the flash-attention change from #15586. Trace how masked attention and runtime backend overrides affect the selected path. Done means memory_required() no longer returns the cheap estimate unless the actual attention path is flash, with coverage for both reported gaps.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.