Comfy-Org / Comfy-Org/ComfyUI

Minimax H3 Ref2VA voice cloning produces gibberish dialogue despite Kijai's fix

Open
#16,155 8 comments 0 reactions 0 assignees View on GitHub
Potential Bug
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Expected Behavior

The character should only speak what is inside the <d></d> tags at the specified time.

### Actual Behavior

The character speaks what is inside the <d></d> tags, but she also speaks gibberish after that (and sometimes even before her actual dialogue). The character doesn't speak at the specified time.

### Steps to Reproduce

Just run the workflow with the reference image and audio clip below, I used the following flags: --windows-standalone-build --cache-none --disable-api-nodes --disable-pinned-memory --cuda-device 0 --use-ck-attention --disable-all-custom-nodes

Audio clip for cloning (I tried to upload it directly on github, but it kept saying it failed!):
https://files.catbox.moe/pcuxvg.wav

Workflow:

https://github.com/user-attachments/assets/af59a33b-2853-4dc1-8413-5b45fef1cdbf

Reference image: Image

Ref2VA workflow WITHOUT voice cloning (this actually works great, as expected):

https://github.com/user-attachments/assets/15f7e7be-29da-456d-97af-2f9f7de61c32

### Debug Logs

```powershell
#### LOGS FOR REF2VA WITH VOICE CLONING
.\python_embeded\python.exe -s ComfyUI\main.py --windows-standalone-build --cache-none --disable-api-nodes --disable-pinned-memory --cuda-device 0 --use-ck-attention --disable-all-custom-nodes
←[32m[INFO]←[0m setup plugin alembic.autogenerate.schemas
←[32m[INFO]←[0m setup plugin alembic.autogenerate.tables
←[32m[INFO]←[0m setup plugin alembic.autogenerate.types
←[32m[INFO]←[0m setup plugin alembic.autogenerate.constraints
←[32m[INFO]←[0m setup plugin alembic.autogenerate.defaults
←[32m[INFO]←[0m setup plugin alembic.autogenerate.comments
[INFO] Set cuda device to: 0
[INFO] Found comfy_kitchen backend eager: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_embedding', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_mxfp8', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'int8_linear', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_mxfp8', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'sol_attn', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend cuda: {'available': True, 'disabled': False, 'unavailable_reason': None, 'capabilities': ['adaln', 'apply_rope', 'apply_rope1', 'apply_rope1_', 'apply_rope_', 'apply_rope_split_half', 'apply_rope_split_half1', 'apply_rope_split_half1_', 'apply_rope_split_half_', 'convrot_w4a4_linear', 'dequantize_convrot_w4a4_weight', 'dequantize_int8_convrot_weight', 'dequantize_int8_convrot_weight_dtype', 'dequantize_int8_simple', 'dequantize_int8_simple_dtype', 'dequantize_nvfp4', 'dequantize_per_tensor_fp8', 'dequantize_w4a8_int8_weight', 'gemv_awq_w4a16', 'int8_linear', 'na3d', 'prepare_int4_weight_for_int8_linear', 'quantize_and_rotate_rowwise', 'quantize_convrot_w4a4_weight', 'quantize_int8_convrot_weight', 'quantize_int8_rowwise', 'quantize_int8_tensorwise', 'quantize_mxfp8', 'quantize_nvfp4', 'quantize_per_tensor_fp8', 'quantize_svdquant_w4a4', 'quantize_w4a8_int8_weight', 'rms_adaln', 'rms_rope', 'rms_rope1', 'rms_rope1_', 'rms_rope_', 'rms_rope_split_half', 'rms_rope_split_half1', 'rms_rope_split_half1_', 'rms_rope_split_half_', 'rotate_int8_convrot_weight', 'scaled_mm_nvfp4', 'scaled_mm_svdquant_w4a4', 'sol_attn', 'stochastic_rounding_fp8', 'w4a8_int8_linear']}
[INFO] Found comfy_kitchen backend triton: {'available': False, 'disabled': True, 'unavailable_reason': "ImportError: No module named 'triton'", 'capabilities': []}
[INFO] Found comfy_kitchen backend hip: {'available': False, 'disabled': False, 'unavailable_reason': 'PyTorch ROCm/HIP runtime not available', 'capabilities': []}
[INFO] Checkpoint files will always be loaded safely.
[INFO] Total VRAM 8192 MB, total RAM 32552 MB
[INFO] pytorch version: 2.13.0+cu130
[INFO] Set vram state to: NORMAL_VRAM
[INFO] Device: cuda:0 NVIDIA GeForce RTX 2070 Super : cudaMallocAsync
[INFO] Using async weight offloading with 2 streams
[INFO] Using pytorch attention
[INFO] Using Comfy Kitchen attention
[INFO] aimdo: src-win/cuda-detour.c:38:INFO:aimdo_setup_hooks: installing 6 hooks
[INFO] aimdo: src/control.c:262:INFO:comfy-aimdo NVML pressure enabled
[INFO] aimdo: src-win/shmem-detect.c:80:INFO:comfy-aimdo WDDM adapter match: NVIDIA GeForce RTX 2070 Super runtime_luid=00000001:1086e45e dxgi_luid=00000001:1086e45e
[INFO] aimdo: src/control.c:277:INFO:comfy-aimdo inited for GPU: NVIDIA GeForce RTX 2070 Super (VRAM: 8191 MB)
[INFO] DynamicVRAM support detected and enabled
[INFO] Python version: 3.13.14 (tags/v3.13.14:fd17997, Jun 10 2026, 13:03:48) [MSC v.1944 64 bit (AMD64)]
[INFO] ComfyUI version: 0.34.0
[INFO] comfy-aimdo version: 0.5.2
[INFO] comfy-kitchen version: 0.2.33
[INFO] comfyui-frontend-package version: 1.51.9
[INFO] comfyui-workflow-templates version: 0.11.55
[INFO] comfyui-embedded-docs version: 0.5.11
[INFO] comfy-kitchen version: 0.2.33
[INFO] comfy-aimdo version: 0.5.2
[INFO] [Prompt Server] web root: C:\Stuff\ComfyUI_windows_portable_cu130\python_embeded\Lib\site-packages\comfyui_frontend_package\static
[INFO] Asset seeder disabled
[INFO] No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
[INFO] Skipping loading of custom nodes
[INFO] Context impl SQLiteImpl.
[INFO] Will assume non-transactional DDL.
[INFO] Disabling intermediate node cache.
[INFO] Starting server

[INFO] To see the GUI go to: http://127.0.0.1:8188
[INFO] got prompt
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float32
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.float16
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load MiniMaxH3VideoVAE
[INFO] Model MiniMaxH3VideoVAE prepared for dynamic VRAM loading. 4965MB Staged. 0 patches attached. Force pre-loaded 128 weights: 348 KB.
[INFO] Requested to load MiniMaxH3AudioVAE
[INFO] Model MiniMaxH3AudioVAE prepared for dynamic VRAM loading. 576MB Staged. 0 patches attached. Force pre-loaded 401 weights: 539 KB.
[INFO] Requested to load MiniMaxH3TEModel_
[INFO] Model MiniMaxH3TEModel_ prepared for dynamic VRAM loading. 49118MB Staged. 0 patches attached. Force pre-loaded 310 weights: 1272 KB.
[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: asym_w4a8_int8, convrot_w4a4, int8_tensorwise , emulated ops: nvfp4, mxfp8, float8_e4m3fn, float8_e5m2
[INFO] model weight dtype torch.bfloat16, manual cast: torch.float32
[INFO] model_type FLOW_AV
[INFO] Requested to load MiniMaxH3
[INFO] 0 models unloaded.
[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading. 21603MB Staged. 0 patches attached. Force pre-loaded 210 weights: 1175 KB.
0%| | 0/20 [00:00

Contributor guide

Open the contributing guide

Research direction

Start by running the linked Ref2VA workflow with voice cloning, using the supplied reference audio, image, flags, and debug logs; compare it with the linked workflow without voice cloning. Trace the MiniMaxH3 and MiniMaxH3AudioVAE execution indicated in the logs. Done means the character speaks only the dialogue inside the tags at the specified time without gibberish.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.