lmstudio-ai / lmstudio-ai/mlx-engine
Error in iterating prediction stream: RuntimeError: [metal::malloc] Resource limit (499000) exceeded.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
Running glm-4.7-flash on mac mini m4 pro, 48Gb RAM. Set context to 200k tokens. And during long chain of thoughts it stops with error `Error in iterating prediction stream: RuntimeError: [metal::malloc] Resource limit (499000) exceeded.`
LM Studio v0.3.39
Runtime:
LM Studio MLX v0.44.1
Contributor guide
Research direction
Start by reproducing the failure while iterating the prediction stream with glm-4.7-flash, a 200k-token context, and the reported LM Studio MLX v0.44.1 setup on Apple hardware. Trace the Metal allocation error and determine whether it is caused by the engine or the runtime; done means the cause is confirmed and the reported configuration no longer fails or has a documented limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100