BF16 WeightHooks can produce black output after model offload
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
## Problem
A mapped model `WeightHook` can produce an all-black image on a BF16 Krea2
checkpoint even though the prompt completes successfully and the same LoRA
works through `LoraLoaderModelOnly`.
This was found while following up #14382, but it is independent of the
quantized state-key fix in #14413 and the setter/cache issue in #15530.
## Reproduction
With DynamicVRAM enabled, compare the same public LoRA and seed:
1. No LoRA: normal image.
2. `LoraLoaderModelOnly`: normal LoRA result.
3. `CreateHookLoraModelOnly -> SetClipHooks`: successful request but
black/non-finite output.
A mapped zero-valued patch also fails, while an unmatched zero-patch Hook is
normal. Patch calculation and the first layer output remain finite. Intercepting
the final write or disabling pinned asynchronous offload avoids the failure.
## Expected behavior
Hook writeback after model preparation/offload must preserve a valid storage
and transfer lifecycle. A mapped zero-valued Hook should match the no-LoRA
control, and a non-zero Hook should remain finite and nonblack.
## Validation
The local candidate keeps plain-weight Hook transfers synchronous and restores
declared model dtype casting. A mapped BF16 subset then completes two
consecutive prompts without the black image.
The full public BF16 Hook exceeds the available 30 GiB host-memory envelope on
the current test system and is not claimed by this report.
Contributor guide
Research direction
Start with the mapped model WeightHook path exercised by CreateHookLoraModelOnly -> SetClipHooks, then reproduce with DynamicVRAM while comparing plain-weight transfers and pinned asynchronous offload. Done means mapped BF16 zero-valued Hooks match the no-LoRA control and non-zero Hooks remain finite and nonblack across two consecutive prompts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100