mudler / mudler/vllm.cpp

fix(SAMPLE-LOGPROBS): upstream now accepts logprobs=-1 on /v1/completions, which falsifies a divergence this tree argued for deliberately

Open
#2,690 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: SAMPLE-LOGPROBS (.agents/engine-matrix.md:142, DONE)

Found by wave PORTQ-5 (#2679) re-deriving PORT-NOW entry [191], upstream 5c9ff5366b vllm#46175. Nothing was executed.

What upstream did

CompletionRequest.check_logprobs stops rejecting logprobs == -1 (the whole-vocabulary sentinel) and changes the message to ​`logprobs` must be a positive value or -1.

What this tree does, and why it matters

src/vllm/entrypoints/openai/protocol.cpp:372-375 rejects every negative value:

if (auto lp = j.find("logprobs");
    lp != j.end() && !lp->is_null() && lp->is_number() && lp->get<int>() < 0) {
  throw std::invalid_argument("`logprobs` must be a positive value.");
}

That matches the pin exactly (git show 5559679229:vllm/entrypoints/openai/completion/protocol.py:498), and it is deliberate: the comment at :366-372 says the completion endpoint has no -1 sentinel and that this "closes the divergence .agents/specs/logprobs-all-sentinel.md records". Upstream has now moved to the other side of that argument.

The rest of the stack is already -1-safe, so this is a one-site change:

  • src/vllm/sampling_params.cpp:121-123 accepts -1.
  • src/vllm/v1/worker/gpu/input_batch.cpp:301 widens it to vocab_size.
  • The chat endpoint already admits -1 (protocol.cpp:551-555).

The stated reason for diverging does not survive

The comment justifies the refusal by saying -1 would emit empty top_logprobs maps, because BuildCompletionLogProbs breaks on idx > num_output_top_logprobs (src/vllm/entrypoints/openai/serving_utils.cpp:143). That is true here — but upstream's own completion serving loop keeps nothing under -1 either, so the two sides are already behaviourally identical and no serving_utils.cpp change is owed. Only the validator.

Size

~5 lines of product code. Two existing negative tests assert the old message (tests/vllm/entrypoints/openai/test_protocol.cpp:729,733) and must be re-pointed.

Record edits owed in the same change, or the tree keeps a comment asserting the opposite of its code: the divergence comment at protocol.cpp:366-372, and the Scope paragraph of .agents/specs/logprobs-all-sentinel.md.

One thing this issue cannot settle

.agents/specs/logprobs-all-sentinel.md names an issue number as the owner of exactly this validator (twice, at its lines 28 and 46). That number does not resolve through gh api today. Several other numbers cited across .agents/ behave the same way while their neighbours resolve normally, so whether those issues were deleted, are invisible to this token, or the API is at fault is undetermined — treat it as REMOTE_UNVERIFIED, not as proof of a dangling reference.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the validator and divergence comment in src/vllm/entrypoints/openai/protocol.cpp, then read the two affected cases in tests/vllm/entrypoints/openai/test_protocol.cpp. Compare the existing completion behavior with the cited sampling and worker paths; no serving_utils.cpp change is indicated. Done means the validator, negative-value tests, and the related .agents records consistently reflect the upstream sentinel behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, documentation, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.