Krea2 Prompt Enhancement causes NaN when enabling Dynamic VRAM on a gfx1030 (RDNA2) card.

I noticed that I can enable dynamic vram on my card and it works well.
However, it creates NaN conversion errors in the model run when I had a prompt enhancement (the default krea2 workflow) on.

I tried to debug it a bit with CoPilot and apparently it happens on Block 26.
HOWEVER the same model / prompt / workflow works fine without prompt enhancement. It also works fine WITH prompt enhancement but WITHOUT dynamic VRAM - however, just once in the latter case and it OOMs for a second gen (or immediately if I try to use fp16 models).

My hunch is that something goes wrong during the offloading of the text encoder / loading of the model so that it ends up pointing to invalid data, resulting in the NaN.

I'll try to work my way through this issue, but so far no cigar. In case anyone has an idea, it'd be much appreciated.

Reproduce the default Krea2 workflow on the gfx1030/RDNA2 card with Dynamic VRAM and prompt enhancement enabled, then compare it with each feature disabled. Start by tracing Block 26 and the text-encoder/model offloading sequence; done means the combined workflow completes without NaN conversion errors while retaining the memory savings that avoid OOM.

Written by the indexing model from the issue text.