lmstudio-ai / lmstudio-ai/lmstudio-python
Correct way to access the logit_bias feature
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 875
- Forks
- 166
- PR merge metrics
- No merged PRs in 30d
Description
It seems the OpenAI API supports the attribute .logit_bias, which allows us to encourage or discorage / ban certain words: https://help.openai.com/en/articles/5247780-using-logit-bias-to-alter-token-probability-with-the-openai-api and the lmstudio API also supports it:
https://lmstudio.ai/docs/app/api/endpoints/openai
However, when I try to use it, it seems to have no effect.
For instance, the word "time" for Qwen3-14b is token no. 1678:
token = a.tokenize("time")
print(token)
[1678]
But if I try to pass logit_bias to Qwen 3, it ignores logit_bias completely:
self.model.act(
chat_object,
tool_list, # Pass the actual tools list
config={
"temp": 0.2,
"maxTokens": MAX_HISTORY_LENGTH,
"logit_bias": {"1678":-100},
},
on_prediction_fragment=self._stream_response,
)
[USER (10:08 AM)]: Complete this sentence: "Once upon a..."
[AI (10:08 AM)]: Once upon a time, there lived a curious little girl who loved exploring the enchanted forest behind her cottage.
Is this the right way to access logit_bias? Or is it a specific model issue?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the self.model.act call and its config handling, using the Qwen3-14b reproduction with token 1678 as the test case. Compare the supplied logit_bias shape with the LM Studio and OpenAI API behavior, then establish whether the SDK applies it or whether the limitation is model-specific; done means the expected behavior is reproduced or clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100