lmstudio-ai / lmstudio-ai/mlx-engine
Support for 1bit models
Open
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 133
- Avg merge
- 21h 6m
- Merged PRs (30d)
- 1
Description
I just downloaded https://huggingface.co/prism-ml/Bonsai-8B-mlx-1bit and got
`Error when loading model: ValueError: [quantize] The requested number of bits 1 is not supported. The supported bits are 2, 3, 4, 5, 6 and 8.
`
I'm on macos, arm, lm studio version 0.4.8
Contributor guide
Research direction
Start by reproducing the loading error with the Bonsai-8B-mlx-1bit model on macOS ARM using LM Studio 0.4.8. Trace the model-loading path where the requested quantization bits are validated, then verify that the model loads successfully and that existing 2-, 3-, 4-, 5-, 6- and 8-bit models still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100