Z-Image Turbo bf16: non-finite output on warm rerun on ROCm gfx1201 (RX 9070 XT), cold run valid
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
On AMD RX 9070 XT (gfx1201, RDNA4) with ROCm, Z-Image Turbo bf16 completes a cold run with valid output, but the first warm rerun of the same prompt produces an image containing non-finite values (NaN/Inf).
Environment:
- GPU: AMD Radeon RX 9070 XT, gfx1201, 16 GB
- torch 2.12.0+rocm7.14.0, HIP 7.14.60850, Linux
- ComfyUI at commit b78cec879b9460d5cb25228a83a942fb78d2cd24
- Model: Comfy-Org/z_image_turbo split files at HF revision 08d04455279082882deaabc8d0d09fc914c071e1 (z_image_turbo_bf16.safetensors diffusion, qwen_3_4b.safetensors text encoder, ae.safetensors VAE), all sha256-verified
Reproduction: standard Z-Image Turbo text-to-image graph (CheckpointLoader-equivalent split loaders, 1024x1024, 8 steps, cfg 1.0, fixed seed), run once (cold, output valid), then rerun the identical prompt (warm run 0): the decoded image contains non-finite values. Reproduced 2/2 on this hardware across two fully independent clones and virtual environments, weeks apart. Server ran default settings: NORMAL_VRAM, DynamicVRAM, PyTorch attention, native component staging.
Not reproduced on CUDA (same graph and artifacts are clean on NVIDIA lanes) and not memory pressure: device peak was well under 16 GB and host RAM under half of 62 GB. A separate intermittent single-run non-finite was also observed once on Intel XPU (Arc B570) with SDXL+LoRA, so this may be a non-CUDA backend numerical issue rather than gfx1201-specific, but only the gfx1201 Z-Image case is deterministic for us.
Caveat: measured at the pinned commit above for benchmarking purposes; not yet retested on current master. Happy to rerun a candidate fix on the same hardware.
Contributor guide
Research direction
Start by reproducing the standard Z-Image Turbo split-loader graph on current master with the stated RX 9070 XT, fixed seed, and identical cold/warm runs. Compare where non-finite values first appear and validate any candidate fix against the pinned environment; done means the warm rerun produces a finite decoded image without regressing the valid cold run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100