lmstudio-ai / lmstudio-ai/mlx-engine

The model has crashed when continuously requested

Open
#157 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
133
Avg merge
21h 6m
Merged PRs (30d)
1

Description

Hello, while using LM Studio, I observed that my currently deployed models (Qwen2.5-VL-32B-Instruct, Qwen3-235B-A22B, and DeepSeek R1, all in MLX format*) consistently crash during sustained inference calls. My LM Studio version is 0.3.15 (Build 11). Below is the error log from one of the failures:
2025-05-07 18:19:54 [DEBUG]
Received request: POST to /v1/chat/completions with body {
"messages": [
{
"role": "user",
"content": "智能助手名称 :新闻分析专家\n主要任务 :请根据提供的新闻内容分析新闻的重要性,并对新闻内容进行权重... ... \"effect\": <新闻对涉及行业或公司构成“重大利空、\"重大利好\"、\"利空\"、\"利好\">\n}"
}
],
"model": "glm-4-32b-0414-abliterated",
"n": 1,
"stream": true,
"temperature": 0.7
}
2025-05-07 18:19:54 [INFO]
[LM STUDIO SERVER] Running chat completion on conversation with 1 messages.
2025-05-07 18:19:54 [INFO]
[LM STUDIO SERVER] Streaming response...
2025-05-07 18:19:54 [DEBUG]
[CacheWrapper][INFO] Trimmed 188 tokens from the prompt cache
2025-05-07 18:19:56 [INFO]
[LM STUDIO SERVER] First token generated. Continuing to stream response..
2025-05-08 06:13:34 [DEBUG]
Fatal Python error: Aborted

Current thread 0x00000072c2367000 (most recent call first):
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx/nn/layers/positional_encoding.py", line 47 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 92 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 125 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 159 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/models/glm4.py", line 178 in __call__
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 357 in _step
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py"
2025-05-08 06:13:34 [DEBUG]
, line 391 in generate_step
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 625 in
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_lm/generate.py", line 636 in stream_generate
File "/Users/tldev/.lmstudio/extensions/backends/vendor/_amphibian/app-mlx-generate-mac-arm64@37/lib/python3.11/site-packages/mlx_engine/generate.py", line 345 in create_generator

Thread 0x00000001fc290c80 (most recent call first):

Extension modules: charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, yaml._yaml, markupsafe._speedups, PIL._imaging, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, PIL._imagingft, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, regex._regex, xxhash._xxhash (total: 31)
2025-05-08 06:13:35 [ERROR]
The model has crashed without additional information. (Exit code: 6). Error Data: n/a, Additional Data: n/a

Contributor guide

Open the contributing guide

Research direction

Start by reproducing sustained streaming chat-completion requests with the listed MLX-format models on LM Studio 0.3.15. Inspect the stack-trace locations in positional_encoding.py and glm4.py, then verify that extended inference no longer exits with the reported Python abort and exit code 6.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.