lmstudio-ai / lmstudio-ai/lmstudio-python
Add option to manually change `PROMPT_PROCESSING_CHUNK_SIZE` in the GUI
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 875
- Forks
- 166
- PR merge metrics
- No merged PRs in 30d
Description
The value of this element is hardcoded to `512` which may be fine for some MacBooks, however for advanced users this one should be available to play with since it can influence prefill speed a bit.
Code reference: https://github.com/lmstudio-ai/mlx-engine/blob/57634aff2fe2b763093624d0cc0bd2e259845b9b/mlx_engine/cache_wrapper.py#L15
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading mlx_engine/cache_wrapper.py at the referenced line to understand how PROMPT_PROCESSING_CHUNK_SIZE is set. Then trace the available GUI configuration entry points; done means advanced users can change this value in the GUI and the selected setting is used during processing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, desktop
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100