Comfy-Org / Comfy-Org/ComfyUI

In the latest comfyui 0.33.3 version (including the dev version), Llama c pp can't recognise mmproj models

Open
#15,859 2 comments 0 reactions 0 assignees View on GitHub
User Support
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)

### Your question

[ERROR] !!! Exception during processing !!! Image input detected, but the loaded model is not configured with a mmproj module.
[ERROR] Traceback (most recent call last):
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 306, in process_inputs
result = f(**inputs)
File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-llama-cpp_vlm\nodes.py", line 519, in process
raise ValueError("Image input detected, but the loaded model is not configured with a mmproj module.")
ValueError: Image input detected, but the loaded model is not configured with a mmproj module.

### Logs

```powershell

```

### Other

_No response_

Contributor guide

Open the contributing guide

Research direction

Start in custom_nodes/ComfyUI-llama-cpp_vlm/nodes.py around line 519 and trace how the model and mmproj module are loaded before process() handles image input. Reproduce the failure with ComfyUI 0.33.3 and an mmproj model; done means the model is recognized with its mmproj module and the shown exception no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.