alibaba / alibaba/MNN

Qwen2.5-Omni-3B/7B 在最新代码上opencl上推理崩溃

Open
#4,802 12 comments 0 reactions 1 assignee Claimed by @Qxinyu View on GitHub
backend:opencl module:llm type:bug
Dominant language
C++
Stars
16.1k
Forks
2.4k
Avg merge
2d 22h
Merged PRs (30d)
56

Description

OpenCL backend 在某个 fp16 buffer kernel 编译阶段失败,然后空指针继续执行导致 Segmentation fault

模型转换
`aiuser@sw002:~/Rye/workspace_0302/mnn_361/omni/MNN$ python3 transformers/llm/export/llmexport.py --path /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/ --export mnn --quant_bit 4 --quant_block 128 --dst_path /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/ --transformer_fuse --mnnconvert "./build/MNNConvert"
[transformers] Model config: tts_text_start_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151860. This may result in unexpected behavior.
[transformers] Model config: tts_text_end_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151861. This may result in unexpected behavior.
[transformers] Model config: tts_text_pad_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151859. This may result in unexpected behavior.
[transformers] Model config: vision_start_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151652. This may result in unexpected behavior.
[transformers] Model config: vision_end_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151653. This may result in unexpected behavior.
[transformers] Model config: audio_start_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151647. This may result in unexpected behavior.
[transformers] Model config: audio_end_token_id must be `None` or an integer within the vocabulary (between 0 and 8447), got 151648. This may result in unexpected behavior.
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2543/2543 [00:26<00:00, 95.65it/s]
[transformers] Qwen2_5OmniForConditionalGeneration LOAD REPORT from: /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/
Key | Status | Details
---------------------------------------------------+------------+--------
token2wav.code2wav_dit_model.rotary_embed.inv_freq | UNEXPECTED |

Notes:
- UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
✅ Done load pretrained model /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/[58.08 s]
✅ Done export talker to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/talker.onnx[14.44 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/talker.mnn[16.00 s]
Can't open file:/sys/devices/system/cpu/cpufreq/ondemand/affected_cpus
CPU Group: [ 67 27 93 65 5 37 75 47 19 85 57 29 95 55 7 39 101 10 77 49 20 87 59 30 97 69 43 21 88 31 98 41 13 51 23 61 1 33 71 9 15 81 53 25 91 63 3 35 73 45 17 83 66 26 92 64 4 36 74 46 18 84 56 28 94 54 6 38 100 76 48 86 58 96 68 8 102 11 42 40 103 12 79 50 22 89 60 0 32 99 70 78 14 80 52 24 90 62 2 34 72 44 16 82 ], 800000 - 3400000
The device supports: i8sdot:0, fp16:0, i8mm: 0, sve2: 0, sme2: 0
✅ Done export token2wav.predit to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/predit.onnx[ 0.23 s]
✅ Done export token2wav.dit to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/dit.onnx[16.49 s]
✅ Done export token2wav.bigvgan to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/bigvgan.onnx[11.32 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/predit.mnn[ 0.20 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/dit.mnn[16.43 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/bigvgan.mnn[ 8.22 s]
✅ Done export visual to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/visual.onnx[15.31 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/visual.mnn[16.96 s]
✅ Done export audio to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/audio.onnx[30.04 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/audio.mnn[15.01 s]
✅ Done export embedding to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn//embeddings_bf16.bin[ 0.59 s]
✅ Done export onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/onnx/llm.onnx[ 3.06 s]
✅ Done convert onnx model to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/llm.mnn[ 1.06 s]
Load LayerNorm data: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1811/1811 [00:00<00:00, 148351.33it/s]
Quant weights: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1811/1811 [00:42<00:00, 42.76it/s]
✅ Done quant model weight to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/llm.mnn.weight[64.61 s]
✅ Done export tokenizer to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn/tokenizer.mtok[ 1.18 s]
✅ Done export config to /data/home/aiuser/Rye/workspace_0302/model/Qwen2.5-Omni-3B/Qwen2.5-Omni-3B-mnn//llm_config.json[ 0.00 s]`

模型推理
`./llm_demo Qwen2.5-Omni-3B-mnn/config.json mnn_vl.txt
config path is Qwen2.5-Omni-3B-mnn/config.json
CPU Group: [ 1 2 3 4 ], 799999 - 1799998
CPU Group: [ 7 8 ], 799830 - 2199850
CPU Group: [ 5 6 ], 799830 - 2299829
CPU Group: [ 9 10 ], 799830 - 2499787
CPU Group: [ 0 11 ], 799830 - 2600083
The device supports: i8sdot:1, fp16:1, i8mm: 1, sve2: 1, sme2: 0
Can't open file:tmp_dd5a0d035964ac64/mnn_cachefile.bin
Load Cache file error.
Can't open file:tmp_dd5a0d035964ac64/mnn_cachefile.bin
Load Cache file error.
Can't open file:tmp_dd5a0d035964ac64/mnn_cachefile.bin
Load Cache file error.
main, 321, cost time: 3212.111084 ms
Prepare for tuning opt Begin
Prepare for tuning opt End
main, 330, cost time: 0.011000 ms
prompt file is mnn_vl.txt
It looks like there's an object with some text on it. The text seems to be "X" and there are some other letters and numbers, but it's hard to make out all of it clearly. It's a bit hard to tell exactly what it is just from the image. If you have more details or context about this image, it might help to figure it out better. What do you think it could be?
Program build log: :39:28: error: loading directly from pointer to type 'const __global half' requires cl_khr_fp16. Use vector data load builtin functions instead
output_ptr[output_offset]=input_ptr[input_offset];
^

error: Compiler frontend failed (error code )

Build program failed, err:-11 !
programName.c_str()=s buffer_convert_buf in buildKernelWithCache, 652
Segmentation fault`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.