The blockwise of 64 is not supported. (bitsandbytes, radeon)

Used:

• main branch of forge.
• ROCM 6.2 (hardware: Radeon RX6600).
• bitsandbytes-0.44.1.dev0-py3-none-manylinux_2_24_x86_64.whl
• linux ubuntu 22.04 (docker image prebuild with ROCM)

When I try to run image generation with model flux1-dev-bnb-nf4-v2.safetensors I have got the following error.

Stacktrace:

  0%|                                                                                                                                                               | 0/20 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/rad/stable-diffusion-webui-forge/modules_forge/main_thread.py", line 30, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "/home/rad/stable-diffusion-webui-forge/modules/txt2img.py", line 131, in txt2img_function
    processed = processing.process_images(p)
  File "/home/rad/stable-diffusion-webui-forge/modules/processing.py", line 842, in process_images
    res = process_images_inner(p)
  File "/home/rad/stable-diffusion-webui-forge/modules/processing.py", line 990, in process_images_inner
    samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
  File "/home/rad/stable-diffusion-webui-forge/modules/processing.py", line 1387, in sample
    samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
  File "/home/rad/stable-diffusion-webui-forge/modules/sd_samplers_kdiffusion.py", line 238, in sample
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "/home/rad/stable-diffusion-webui-forge/modules/sd_samplers_common.py", line 278, in launch_sampling
    return func()
  File "/home/rad/stable-diffusion-webui-forge/modules/sd_samplers_kdiffusion.py", line 238, in <lambda>
    samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/k_diffusion/sampling.py", line 129, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/modules/sd_samplers_cfg_denoiser.py", line 199, in forward
    denoised, cond_pred, uncond_pred = sampling_function(self, denoiser_params=denoiser_params, cond_scale=cond_scale, cond_composition=cond_composition)
  File "/home/rad/stable-diffusion-webui-forge/backend/sampling/sampling_function.py", line 362, in sampling_function
    denoised, cond_pred, uncond_pred = sampling_function_inner(model, x, timestep, uncond, cond, cond_scale, model_options, seed, return_full=True)
  File "/home/rad/stable-diffusion-webui-forge/backend/sampling/sampling_function.py", line 303, in sampling_function_inner
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
  File "/home/rad/stable-diffusion-webui-forge/backend/sampling/sampling_function.py", line 273, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
  File "/home/rad/stable-diffusion-webui-forge/backend/modules/k_model.py", line 45, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/backend/nn/flux.py", line 418, in forward
    out = self.inner_forward(img, img_ids, context, txt_ids, timestep, y, guidance)
  File "/home/rad/stable-diffusion-webui-forge/backend/nn/flux.py", line 375, in inner_forward
    img = self.img_in(img)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/rad/stable-diffusion-webui-forge/backend/operations.py", line 367, in forward
    return functional_linear_4bits(x, self.weight, self.bias)
  File "/home/rad/stable-diffusion-webui-forge/backend/operations_bnb.py", line 12, in functional_linear_4bits
    out = bnb.matmul_4bit(x, weight.t(), bias=bias, quant_state=weight.quant_state)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 601, in matmul_4bit
    return MatMul4Bit.apply(A, B, out, bias, quant_state)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/torch/autograd/function.py", line 575, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 522, in forward
    output = torch.nn.functional.linear(A, F.dequantize_4bit(B, quant_state).to(A.dtype).t(), bias)
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/bitsandbytes/functional.py", line 1065, in dequantize_4bit
    return backends[A.device.type].dequantize_4bit(
  File "/home/rad/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/bitsandbytes/backends/cuda.py", line 563, in dequantize_4bit
    raise ValueError(
ValueError: The blockwise of 64 is not supported. Supported values: [2048, 4096, 1024, 512, 256, 128]
The blockwise of 64 is not supported. Supported values: [2048, 4096, 1024, 512, 256, 128]

This is caused due bitsandbytes does not support 64 blocks on HIP devices :

        # Some AMD GPUs have warpsize 64
        # Set min blocksize to 128 (~warpsize 64 in kernel) for HIP
        if HIP_ENVIRONMENT:
            supported_blocksizes = supported_blocksizes[:-1]
        if quant_state.blocksize not in supported_blocksizes:
            raise ValueError(
                f"The blockwise of {quant_state.blocksize} is not supported. Supported values: {supported_blocksizes}",
            )

Reproduce the Flux image-generation failure on the stated Radeon/ROCm setup, then read backend/operations_bnb.py and the referenced bitsandbytes functional.py block-size check. Determine the compatibility change needed for block size 64 and verify that the same generation path completes without this ValueError.