lmstudio-ai / lmstudio-ai/lmstudio-python

Correct way to access the logit_bias feature

Open
#87 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
875
Forks
166
PR merge metrics
No merged PRs in 30d

Description

It seems the OpenAI API supports the attribute .logit_bias, which allows us to encourage or discorage / ban certain words: https://help.openai.com/en/articles/5247780-using-logit-bias-to-alter-token-probability-with-the-openai-api and the lmstudio API also supports it:
https://lmstudio.ai/docs/app/api/endpoints/openai

However, when I try to use it, it seems to have no effect.
For instance, the word "time" for Qwen3-14b is token no. 1678:

token = a.tokenize("time")
print(token)
[1678]

But if I try to pass logit_bias to Qwen 3, it ignores logit_bias completely:

self.model.act(
chat_object,
tool_list, # Pass the actual tools list
config={
"temp": 0.2,
"maxTokens": MAX_HISTORY_LENGTH,
"logit_bias": {"1678":-100},
},
on_prediction_fragment=self._stream_response,
)

[USER (10:08 AM)]: Complete this sentence: "Once upon a..."

[AI (10:08 AM)]: Once upon a time, there lived a curious little girl who loved exploring the enchanted forest behind her cottage.

Is this the right way to access logit_bias? Or is it a specific model issue?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the self.model.act call and its config handling, using the Qwen3-14b reproduction with token 1678 as the test case. Compare the supplied logit_bias shape with the LM Studio and OpenAI API behavior, then establish whether the SDK applies it or whether the limitation is model-specific; done means the expected behavior is reproduced or clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.