lmstudio-ai / lmstudio-ai/mlx-engine

We hope to upgrade the mlx-vlm version to 0.6.2 or above to resolve the issue of VLM lacking the qwen3_5_moe_vision type, which prevents the Holo 3.1 model from being used.

Open Beginner friendly
#339 3 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
133
Avg merge
21h 6m
Merged PRs (30d)
1

Description

# Problem

We hope to upgrade the mlx-vlm version to 0.6.2 or above to resolve the issue of VLM lacking the qwen3_5_moe_vision type, which prevents the Holo 3.1 model from being used.

# Related URL

https://github.com/Blaizzy/mlx-vlm/pull/1302
https://github.com/Blaizzy/mlx-vlm/releases/tag/v0.6.2

# Error Log

```log
Error when loading model: Exception: Encountered fatal exception in the backend generation thread: Traceback (most recent call last):
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 346, in _generate_with_exception_handling
self._generate()
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 582, in _generate
self._load_model()
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_engine/model_kit/batched_vision/model_kit.py", line 210, in _load_model
self.model = mlx_vlm.utils.load_model(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/utils.py", line 484, in load_model
model = model_class.Model(model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/models/qwen3_5_moe/qwen3_5_moe.py", line 17, in __init__
self.vision_tower = VisionModel(config.vision_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mushan/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac14-arm64@30/lib/python3.11/site-packages/mlx_vlm/models/qwen3_vl/vision.py", line 201, in __init__
raise ValueError(f"Unsupported model type: {self.model_type}")
ValueError: Unsupported model type: qwen3_5_moe_vision
```

Contributor guide

Open the contributing guide

Research direction

Inspect the repository's dependency declarations and update mlx-vlm to 0.6.2 or newer. Start from the mlx_vlm.utils.load_model path and the qwen3_5_moe_vision error shown in the report, then verify that the Holo 3.1 model loads without the unsupported-model-type failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.