lmstudio-ai / lmstudio-ai/mlx-engine
We hope to upgrade the mlx-vlm version to 0.6.2 or above to resolve the issue of VLM lacking the qwen3_5_moe_vision type, which prevents the Holo 3.1 model from being used.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
# Problem
We hope to upgrade the mlx-vlm version to 0.6.2 or above to resolve the issue of VLM lacking the qwen3_5_moe_vision type, which prevents the Holo 3.1 model from being used.
# Related URL
https://github.com/Blaizzy/mlx-vlm/pull/1302
https://github.com/Blaizzy/mlx-vlm/releases/tag/v0.6.2
# Error Log
```log
Error when loading model: Exception: Encountered fatal exception in the backend generation thread: Traceback (most recent call last):
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 346, in _generate_with_exception_handling
self._generate()
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 582, in _generate
self._load_model()
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 210, in _load_model
self.model = mlx_vlm.utils.load_model(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/utils.py", line 484, in load_model
model = model_class.Model(model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/models/qwen3_5_moe/qwen3_5_moe.py", line 17, in __init__
self.vision_tower = VisionModel(config.vision_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/models/qwen3_vl/vision.py", line 201, in __init__
raise ValueError(f"Unsupported model type: {self.model_type}")
ValueError: Unsupported model type: qwen3_5_moe_vision
```
Contributor guide
Research direction
Inspect the repository's dependency declarations and update mlx-vlm to 0.6.2 or newer. Start from the mlx_vlm.utils.load_model path and the qwen3_5_moe_vision error shown in the report, then verify that the Holo 3.1 model loads without the unsupported-model-type failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100