Comfy-Org / Comfy-Org/ComfyUI

Trellis2 ROCm Portable 7900XTX Windows - Default Workflow - UnwrapMesh UVs node - HIPBLAS_STATUS_ALLOC_FAILED hipblasDgetrfBatched

Open
#16,124 4 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 10h
Merged PRs (30d)
153

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

Diffuse the mesh texture

### Actual Behavior

UnwrapMesh UVs stage is about adding vertex in the mesh, preparing it for texturing.

ROCm has severe issues with memory allocation, it could have to do with that, i tried with dynamic memory disabled and the issue is the same

RuntimeError: CUDA error: HIPBLAS_STATUS_ALLOC_FAILED when calling `hipblasDgetrfBatched( handle, n, dA_array, ldda, ipiv_array, info_array, batchsize)`

### Steps to Reproduce

ComfyUI portable, run default Trellis 2 workflow with default models. on 7900XTX ComfyUI AMD Windows portable.

### Debug Logs

```powershell
F:\ComfyUI_windows_portable_amd>.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --disable-all-custom-nodes
[INFO] setup plugin alembic.autogenerate.schemas
[INFO] setup plugin alembic.autogenerate.tables
[INFO] setup plugin alembic.autogenerate.types
[INFO] setup plugin alembic.autogenerate.constraints
[INFO] setup plugin alembic.autogenerate.defaults
[INFO] setup plugin alembic.autogenerate.comments
[WARNING] On windows we are currently forcing single GPU mode in ComfyUI due to a Nvidia related issue, if you want to disable this use: --cuda-device all
[INFO] Adding extra search path checkpoints F:\ComfyUI-Models\models
[INFO] Adding extra search path clip F:\ComfyUI-Models\clip
[INFO] Adding extra search path clip_vision F:\ComfyUI-Models\clip_vision
[INFO] Adding extra search path text_encoders F:\ComfyUI-Models\clip
[INFO] Adding extra search path configs F:\ComfyUI-Models\configs
[INFO] Adding extra search path controlnet F:\ComfyUI-Models\controlnet
[INFO] Adding extra search path diffusion_models F:\ComfyUI-Models\models
[INFO] Adding extra search path embeddings F:\ComfyUI-Models\embeddings
[INFO] Adding extra search path loras F:\ComfyUI-Models\loras
[INFO] Adding extra search path upscale_models F:\ComfyUI-Models\upscale_models
[INFO] Adding extra search path vae F:\ComfyUI-Models\vae
[INFO] Adding extra search path background_removal F:\ComfyUI-Models\background_removal
[INFO] Adding extra search path unet F:\ComfyUI-Models\models
[INFO] Adding extra search path vibevoice F:\ComfyUI-Models\vibevoice
Fatal error in launcher: Unable to create process using '"D:\a\ComfyUI\python_embeded\python.exe" "F:\ComfyUI_windows_portable_amd\python_embeded\Scripts\offload-arch.exe" ': The system cannot find the file specified.

[WARNING] offload-arch failed with return code 1
[stderr]
[INFO] Found comfy_kitchen backend triton: {'available': False, 'disabled': True, 'unavailable_reason': "ImportError: No module named 'triton'", 'capabilities': []}
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_embedding', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'int8_linear', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': True, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend hip: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple_dtype', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'int8_linear', 'na3d', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'scaled_mm_svdquant_w4a4', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 24560 MB, total RAM 65367 MB
[INFO] pytorch version: 2.9.1+rocm7.2.1
[INFO] Set: torch.backends.cudnn.enabled = False for better AMD performance.
[INFO] AMD arch: gfx1100
[INFO] ROCm version: (7, 2)
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 AMD Radeon RX 7900 XTX : native
[INFO] Using async weight offloading with 2 streams
[INFO] Enabled pinned memory 26146.0
[INFO] Using pytorch attention
[INFO] Python version: 3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
[INFO] ComfyUI version: 0.34.0
[INFO] comfy-aimdo version: 0.5.2
[INFO] comfy-kitchen version: 0.2.31
[INFO] comfyui-frontend-package version: 1.51.9
[INFO] comfyui-workflow-templates version: 0.11.55
[INFO] comfyui-embedded-docs version: 0.5.10
[INFO] comfy-kitchen version: 0.2.31
[INFO] comfy-aimdo version: 0.5.2
[INFO] [Prompt Server] web root: F:\ComfyUI_windows_portable_amd\python_embeded\Lib\site-packages\comfyui_frontend_package\static
[INFO] Asset seeder disabled
[INFO] No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
[INFO] Skipping loading of custom nodes
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Using RAM pressure cache.
[INFO] Starting server

[INFO] To see the GUI go to: http://127.0.0.1:8188
[INFO] got prompt
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: asym_w4a8_int8, int8_tensorwise, convrot_w4a4 , emulated ops: float8_e5m2, float8_e4m3fn, mxfp8, nvfp4
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] Requested to load BiRefNet
[INFO] loaded completely; 22894.08 MB usable, 419.43 MB loaded, full load: True
[INFO] Requested to load DINOv3ViTModel
[INFO] loaded completely; 22210.81 MB usable, 578.16 MB loaded, full load: True
F:\ComfyUI_windows_portable_amd\ComfyUI\comfy\ops.py:95: UserWarning: Using AOTriton backend for Efficient Attention forward... (Triggered internally at C:/b/pytorch/aten/src/ATen/native/transformers/hip/attention.hip:1452.)
return torch.nn.functional.scaled_dot_product_attention(q, k, v, *args, **kwargs)
[INFO] Requested to load Trellis2
[INFO] loaded completely; 21563.94 MB usable, 5007.09 MB loaded, full load: True
100%|██████████████████████████████████████████████████████████████████████████████████| 12/12 [00:07<00:00, 1.65it/s]
[INFO] Requested to load ShapeVae
[INFO] loaded completely; 16553.97 MB usable, 1045.04 MB loaded, full load: True
[INFO] Requested to load Trellis2
25%|████████████████████▊ | 5/2 30%|████████████████████████▉ | 6/2 35%|█████████████████████████████
40%|█████████████████████████████████▏
45%|█████████████████████████████████████▎ 50%|█████████████████████████████████████████
55%|█████████████████████████████████████████████
60%|█████████████████████████████████████████████████▏
65%|█████████████████████████████████████████████████████▎
70%|█████████████████████████████████████████████████████████▍
75%|█████████████████████████████████████████████████████████████▌
80%|█████████████████████████████████████████████████████████████████▌
85%|█████████████████████████████████████████████████████████████████████▋
90%|█████████████████████████████████████████████████████████████████████████▊ 95%|█████████████████████████████████████████████████████████████████████████████▉ 100%|█████████████████████████████████████████████████████████████████████████████████100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:09<00:00, 2.13it/s]
100%|████████████████████████████████████████████████| 12/12 [04:27<00:00, 22.27s/it]
[INFO] Requested to load Trellis2
100%|████████████████████████████████████████████████| 12/12 [02:26<00:00, 12.24s/it]
[INFO] Requested to load TextureVae
[INFO] loaded completely; 9633.75 MB usable, 904.50 MB loaded, full load: True
[INFO] [GetMeshInfo]
Vertices: 10,217,222 (10.22M)
Faces: 20,301,974 (20.30M)
Attributes: none
[ERROR] !!! Exception during processing !!! CUDA error: HIPBLAS_STATUS_ALLOC_FAILED when calling `hipblasDgetrfBatched( handle, n, dA_array, ldda, ipiv_array, info_array, batchsize)`
[ERROR] Traceback (most recent call last):
File "F:\ComfyUI_windows_portable_amd\ComfyUI\execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "F:\ComfyUI_windows_portable_amd\ComfyUI\execution.py", line 306, in process_inputs
result = f(**inputs)
^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\comfy_api\internal\__init__.py", line 149, in wrapped_func
return method(locked_class, **inputs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\comfy_api\latest\_io.py", line 2046, in EXECUTE_NORMALIZED
to_return = cls.execute(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\comfy_extras\nodes_mesh_postprocess.py", line 2774, in execute
vmapping, indices, uvs = _uv_unwrap(
^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\comfy_extras\nodes_mesh_postprocess.py", line 2644, in _uv_unwrap
lscm_uv = _uv_param.lscm_charts_batch(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\ComfyUI_windows_portable_amd\ComfyUI\comfy_extras\mesh3d\uv_unwrap\parameterize.py", line 338, in lscm_charts_batch
x = torch.linalg.solve(AtA, Atb).cpu().numpy()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: HIPBLAS_STATUS_ALLOC_FAILED when calling `hipblasDgetrfBatched( handle, n, dA_array, ldda, ipiv_array, info_array, batchsize)`
```

### Other

System

## System Info
OS: win32
Python Version: 3.12.10 (tags/v3.12.10:0cc8128, Apr 8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
Embedded Python: true
PyTorch Version: 2.9.1+rocm7.2.1
Arguments: ComfyUI\main.py --windows-standalone-build --disable-all-custom-nodes
RAM Total: 63.83 GB
RAM Free: 47.83 GB
Templates Version: 0.11.55

## Devices
- cuda:0 AMD Radeon RX 7900 XTX : native (cuda)
VRAM Total: 23.98 GB
VRAM Free: 23.84 GB
Torch VRAM Total: 2 MB
Torch VRAM Free: 2 MB

Contributor guide

Open the contributing guide

Research direction

Reproduce the default Trellis2 workflow with custom nodes disabled on the AMD Windows portable setup, focusing on the UnwrapMesh UVs stage. Start with the traceback locations in execution.py and the GPU operation shown at comfy/ops.py:95; done means the workflow completes texture diffusion without the HIPBLAS_STATUS_ALLOC_FAILED error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.