lmstudio-ai / lmstudio-ai/mlx-engine
The model has crashed when continuously requested
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
Hello, while using LM Studio, I observed that my currently deployed models (Qwen2.5-VL-32B-Instruct, Qwen3-235B-A22B, and DeepSeek R1, all in MLX format*) consistently crash during sustained inference calls. My LM Studio version is 0.3.15 (Build 11). Below is the error log from one of the failures:
2025-05-07 18:19:54 [DEBUG]
Received request: POST to /v1/chat/completions with body {
"messages": [
{
"role": "user",
"content": "智能助手名称 :新闻分析专家\n主要任务 :请根据提供的新闻内容分析新闻的重要性,并对新闻内容进行权重... ... \"effect\": <新闻对涉及行业或公司构成“重大利空、\"重大利好\"、\"利空\"、\"利好\">\n}"
}
],
"model": "glm-4-32b-0414-abliterated",
"n": 1,
"stream": true,
"temperature": 0.7
}
2025-05-07 18:19:54 [INFO]
[LM STUDIO SERVER] Running chat completion on conversation with 1 messages.
2025-05-07 18:19:54 [INFO]
[LM STUDIO SERVER] Streaming response...
2025-05-07 18:19:54 [DEBUG]
[CacheWrapper][INFO] Trimmed 188 tokens from the prompt cache
2025-05-07 18:19:56 [INFO]
[LM STUDIO SERVER] First token generated. Continuing to stream response..
2025-05-08 06:13:34 [DEBUG]
Fatal Python error: Aborted
Current thread 0x00000072c2367000 (most recent call first):
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx/nn/layers/positional_encoding.py", line 47 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 92 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 125 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 159 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 178 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 357 in _step
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py"
2025-05-08 06:13:34 [DEBUG]
, line 391 in generate_step
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 625 in
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 636 in stream_generate
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_engine/generate.py", line 345 in create_generator
Thread 0x00000001fc290c80 (most recent call first):
Extension modules: charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, yaml._yaml, markupsafe._speedups, PIL._imaging, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, PIL._imagingft, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, regex._regex, xxhash._xxhash (total: 31)
2025-05-08 06:13:35 [ERROR]
The model has crashed without additional information. (Exit code: 6). Error Data: n/a, Additional Data: n/a
Contributor guide
Research direction
Start by reproducing sustained streaming chat-completion requests with the listed MLX-format models on LM Studio 0.3.15. Inspect the stack-trace locations in positional_encoding.py and glm4.py, then verify that extended inference no longer exits with the reported Python abort and exit code 6.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100