lmstudio-ai / lmstudio-ai/lmstudio-python
Does .chat() support setting context length?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 875
- Forks
- 166
- PR merge metrics
- No merged PRs in 30d
Description
Hello I'm trying to set context length when running this code. Placing the configuration value in different places hasn't worked.
I am able to load the model with manually set context length in the GUI, then the terminal program runs fine using model = lms.llm()
https://lmstudio.ai/docs/python/llm-prediction/chat-completion#example-multi-turn-chat
If I load the config value with the model it doesn't apply the setting when opening .chat()
model = lms.llm("OpenAI-20B-NEO-CODEPlus-Uncensored-IQ4_NL.gguf", config={ "contextLength": 8192 })
If I set it in the chat I get an error. Example code:
chat = lms.Chat(systemprompt, config={ "contextLength": 8192 })
error:
line 59, in <module> chat = lms.Chat(systemprompt, config={ "contextLength": 8192 }) TypeError: Chat.__init__() got an unexpected keyword argument 'config'
Help with this issue would be appreciated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked multi-turn chat example and inspect how lms.llm() and lms.Chat() accept configuration. Reproduce the TypeError with the shown contextLength value and determine the supported configuration path; done means the documented chat flow can set a context length of 8192 without the constructor error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100