lmstudio-ai / lmstudio-ai/mlx-engine
Text-only architectures implemented only in mlx_vlm (e.g. deepseek_v4) fail to load because dispatch checks only for vision_config
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
## Summary
For a text-only model, `mlx_engine/generate.py`'s model-kit selection dispatches purely on `"vision_config" in config_json` (around `load_model`, the `is_batchable()` branch). If that key is absent, it unconditionally calls `mlx_lm_load(model_path, lazy=True)` — even when the architecture in question has no `mlx_lm` implementation at all, but *does* have one in `mlx_vlm`, bundled in the exact same environment.
Concretely, for `deepseek_v4` (a genuinely text-only architecture — no vision component): the bundled environment's `mlx_lm/models/` stops at `deepseek_v32`, so loading fails outright. The *same* bundled environment's `mlx_vlm/models/deepseek_v4/` is fully implemented (including an MTP speculative-decoding drafter), but it's never reached, because the dispatch never considers `mlx_vlm` for a model that lacks `vision_config`.
## Repro
Load any local MLX checkpoint with `model_type: deepseek_v4` (text-only, no `vision_config` key) via `lms load` or the GUI.
Result:
```
ValueError: Model type deepseek_v4 not supported.
At:
.../mlx_lm/utils.py(192): _get_classes
.../mlx_lm/utils.py(348): load_model
.../mlx_lm/utils.py(509): load
.../mlx_engine/generate.py(289): is_batchable
```
## Workaround found
Adding an empty `"vision_config": {}` to the checkpoint's `config.json` routes the same load through the `BatchedVisionModelKit` → `mlx_vlm` path instead, which has the architecture and loads/generates correctly (confirmed end-to-end, including at extended context lengths). `mlx_vlm.utils.load_image_processor` degrades gracefully for a model class with no `ImageProcessor` attribute, so this doesn't require faking any real vision capability — it's purely a dispatch trick, and a fragile one for users who don't know to do it.
## Suggested fix
Before deciding "no `vision_config` → use `mlx_lm`", check whether `mlx_lm` actually has an implementation for the model's `model_type` (a cheap `_get_classes`-style lookup) and fall back to `mlx_vlm` if not, rather than dispatching solely on the presence of a config key that's orthogonal to which package actually implements the architecture. This matters more as new architectures increasingly land in `mlx_vlm` first even when text-only.
## Environment
- LM Studio 0.4.20 (build 1), macOS
- mlx-llm backend (nax-advsimd), engine build `app-mlx-generate-mac26-arm64@33`
- Model: local MLX conversion of `deepseek-ai/DeepSeek-V4-Flash` (`deepseek_v4` architecture, text-only)
- Related: #356 (the same underlying gap also causes the manual context-length control to be capped, since the auto-fit path that would otherwise report a real value is never reached for this architecture either)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in mlx_engine/generate.py around load_model and the is_batchable() dispatch, then inspect the model-type lookup behavior described for mlx_lm and mlx_vlm. Reproduce with a local deepseek_v4 checkpoint lacking vision_config through lms load or the GUI. Done means text-only architectures implemented only in mlx_vlm load without a vision_config workaround, while existing vision-model routing still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100