### Is there an existing issue for this problem?

- [x] I have searched the existing issues

### Install method

Docker image on unRAID server

### Operating system

Linux (unRAID)

### GPU vendor

AMD (ROCm)

### GPU model

AI PRO R9700

### GPU VRAM

32 GiB

### Version number

v6.14.1-post1

### Browser

Edge Version 150.0.4078.48 (Official build) (64-bit)

### System Information

### What happened

[bug]: FP8 and other quantized checkpoints are expanded to full precision on load — resident size is identical regardless of quantization level (ROCm)

Is there an existing issue for this problem?

• [x] I have searched the existing issues

Operating system

Linux (unRAID 7.x, Docker)

GPU vendor

AMD (ROCm)

GPU model

GPU VRAM

32 GB

Version number

6.14.0 and 6.14.1-post1 — identical behaviour on both.

• ghcr.io/invoke-ai/invokeai:6.14.0-rocm — sha256:032161d7873e3d3f13f699fa9a8a10ec3f5a3e86b662f07843f263fd1a245580


• ghcr.io/invoke-ai/invokeai:main-rocm (6.14.1-post1) — sha256:2bb1a68525a8c6e567ee499ac391a8289c469414fc9b9bea74eeb37caa7d70de

Browser

What happened

On ROCm, quantized checkpoints appear to be dequantized to full precision during
loading. The memory saving that quantization is chosen for is lost entirely, and the
resident footprint is the same no matter which quantization is used.

GGUF checkpoints are unaffected — they load at approximately their on-disk size.

Evidence 1: three different Z Image quantizations, one resident size

Three distinct Z Image checkpoints, three different file sizes, all reporting the
same loaded size:

Model ID | Size on disk | Total model size as loaded
-- | -- | --
d9006a28-cc54-49d3-93d9-3e1652d2cdf5 | 6.16 GiB | 11,739.56 MB
00ce5266-f09d-4287-a4ea-d8ae89f56263 | 6.54 GiB | 11,739.56 MB
f50e705c-df8a-496f-9acf-491fba3d885f | 12.57 GiB | 11,739.56 MB

Two independent FP8 Krea-2 checkpoints both land at exactly 24,452.35 MB. The GGUF
build of the same model family stays at its on-disk size.

precision: bfloat16 is set in invokeai.yaml, and ~2x is what FP8 → bf16 expansion
would produce.

Consequence

The model no longer fits in the cache, so it is evicted and re-staged on every
generation:

• GGUF Krea-2 (12.69 GiB resident, ~21 GiB working set with the encoder):
  stays cached, repeated in 0.00s cache hits, 22 s per 960x1344 image.


• FP8 Krea-2 (23.88 GiB resident, ~32 GiB working set): never a single cache
  hit, full re-stage every generation, 82 s per image, and OOM-killed outright at
  container limits below 26 GiB.

The staging phase is host-RAM bound with the GPU idle

Sampling the container cgroup's memory.current and amdgpu's mem_info_vram_used
every 2 seconds during an FP8 load:

13:58:57  sys=  8.2 GiB  vram=  0.6 GiB    <- load begins

13:59:07  sys= 20.3 GiB  vram=  0.6 GiB

13:59:19  sys= 26.0 GiB  vram=  0.6 GiB    <- pinned at container limit

   ... 45 seconds pinned, GPU idle at 0.6 GiB ...

14:00:06  sys= 25.6 GiB  vram=  1.2 GiB    <- transfer to VRAM begins

14:00:12  sys=  6.2 GiB  vram= 20.6 GiB

14:00:16  sys=  2.5 GiB  vram= 26.0 GiB    <- steady state

The weights are fully materialised in system RAM, then copied to VRAM in ~6 seconds.
Steady-state host RAM afterwards is 2.5 GiB. So host RAM must be at least
model-sized during loading even when VRAM is abundant, and the expansion above
doubles what that costs.

What you expected to happen

That an FP8 checkpoint would occupy roughly its on-disk size in memory, and that
selecting a smaller quantization would reduce the memory footprint.

How to reproduce the problem

1. ROCm container image on a system with 32 GB VRAM.


2. Install two quantizations of the same model — e.g. a 6 GiB and a 12 GiB Z Image
   checkpoint, or an FP8 and a Q8_0 GGUF Krea-2.


3. Load each and compare the Total model size value in the
   [MODEL CACHE] Loaded model log line against the file size on disk.


4. The quantized non-GGUF checkpoints report a resident size unrelated to their file
   size.

Possible cause

The ROCm image does not contain rocminfo, so bitsandbytes cannot detect the GPU
architecture and falls back. Logged at every startup:

Could not detect ROCm GPU architecture: [Errno 2] No such file or directory: 'rocminfo'

ROCm GPU architecture detection failed despite ROCm being available.

Could not detect ROCm warp size: [Errno 2] No such file or directory: 'rocminfo'.

Defaulting to 64. (some 4-bit functions may not work!)

If quantized kernels are unavailable because the architecture could not be
identified, dequantizing the weights at load would be a plausible fallback — and
would explain why GGUF (which has its own dequantization path) behaves differently.

This is speculation on my part; I have not instrumented it. But rocminfo being
absent from the official ROCm image looks like a packaging defect regardless.

Environment

MALLOC_MMAP_THRESHOLD_=1048576

DISABLE_PINNED_MEMORY=1

PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:128

HSA_OVERRIDE_GFX_VERSION=12.0.1

HSA_ENABLE_SDMA=0

PYTORCH_TUNABLEOP_ENABLED=0

TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=0

MIOPEN_FIND_MODE=FAST

GPU_DRIVER=rocm

schema_version: 4.0.3

max_cache_ram_gb: 14.0

keep_ram_copy_of_weights: false

device_working_mem_gb: 3.0

model_cache_keep_alive_min: 0

enable_partial_loading: false

force_tiled_decode: true

device: auto

precision: bfloat16

attention_type: torch-sdp

Host: 32 GB system RAM (31 GiB usable), AMD Ryzen 7 3800X (8C/16T), container limited
to 26 GiB and 6 CPUs. Nothing else memory-heavy running. Components in the Krea-2
workflow: Qwen3-VL 4B encoder (8,464.46 MB), Qwen Image VAE (242.03 MB).

What this is not

Recorded so others don't repeat the dead ends I worked through:

• Not a leak. Host RAM returns to 2.5–7.8 GiB after every load; nothing
  accumulates across generations.


• Not a regression. 6.14.0 and 6.14.1-post1 are identical. Earlier versions
  could not be tested — Krea-2 support does not exist in 6.13.x.


• Not fixable via max_cache_ram_gb. Tested at 8.0, 12.0 and 14.0.


• Not glibc fragmentation. The FAQ's MALLOC_MMAP_THRESHOLD_=1048576 workaround
  was in place throughout, with DISABLE_PINNED_MEMORY=1.


• Not a GPU fault. No amdgpu ring timeouts, resets or segfaults in dmesg.
  All OOM kills are CONSTRAINT_MEMCG, contained to the container cgroup, with no
  Python traceback.

Three smaller issues noticed along the way

Happy to split these out.

1. The RAM cache statistics block never updates. Byte-identical across
consecutive generations (hits: 2 / misses: 4 / cached: 1 / cleared: 1) despite each
run loading several models.

2. Model load time is attributed to the denoise node. krea2_denoise was billed
at 65.2 s while the 9 sampling steps took ~20 s (2.26 s/it) and the Loaded model
line reported 3.16 s. The ~45 s of staging is folded into denoise and invisible in
the graph stats — the timing table points at sampling when the cost is in loading.
This made the problem substantially harder to locate.

3. force_tiled_decode: true appeared to have no effect. VRAM still spiked to
31.6 GiB at VAE decode with it enabled. Possibly not wired to the Qwen Image VAE
path.

### What you expected to happen

I expected to be able to run the models.

### How to reproduce the problem

Start the app load any model which is not quantized.

### Additional context

_No response_

### Discord username

C_Dollerup

Start by reproducing the load comparison described for the FP8, quantized Z Image, and GGUF checkpoints, using the precision setting in invokeai.yaml and recording resident memory and cache behavior. Trace the model-loading path responsible for the FP8-to-bfloat16 expansion. Done means quantized checkpoints retain an appropriately reduced memory footprint, avoid repeated restaging, and GGUF behavior remains unaffected.

Written by the indexing model from the issue text.