mudler / mudler/vllm.cpp

port(ENG-PRIORITY-SCHED): the X-Vllm-Priority request header is never read

Open
#2,648 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: ENG-PRIORITY-SCHED

Found by #2647 (PORT-NOW wave
PORTQ-3, tranche 81-120). Nothing was executed: this is a static reading of the
tree at e24ec8bfd against upstream 22170354d8 (vllm#51089).

What is missing

Upstream 22170354d8 adds PRIORITY_HEADER = "X-Vllm-Priority" and a
_get_priority helper on the serving base, so a client can set a request's
scheduling priority from an HTTP header. The chat and completion serving paths
resolve priority through it instead of reading request.priority directly.

This tree reads the body field only. X-Vllm-Priority has zero hits in src/,
include/ and tests/.

What IS here

Everything downstream of the header is already plumbed, which is why this is a
bounded port rather than a feature:

  • the priority policy is a production CLI option —
    src/vllm/entrypoints/openai/server_main.cpp --scheduling-policy priority,
    threaded to SchedulerConfig::policy;
  • the body field is parsed at
    src/vllm/entrypoints/openai/protocol.cpp:385 (GetOr(j, "priority", r.priority));
  • and forwarded to the engine at
    src/vllm/entrypoints/openai/serving_chat.cpp:821 and
    src/vllm/entrypoints/openai/serving_completion.cpp:338.

The gap is only the alternative source. The route handlers in
src/vllm/entrypoints/openai/api_server.cpp hold the httplib::Request, but the
serving layer receives only the parsed request object, so the header never
reaches the place that decides the priority.

Size

Roughly 20-40 lines: read and int-parse the header at the route handler, fall
back silently to the body field on a malformed value exactly as upstream does,
and thread it into the serving call. Plus a socket-level test that a request with
the header admits at the header's priority and one with a malformed header falls
back.

Why it matters

ENG-PRIORITY-SCHED is GATING and its own scope line names "priority
plumbing (Request/EngineCoreRequest/OpenAI field)". This is the one input source
of that plumbing upstream has and this tree does not, and it is a user-visible
API contract difference rather than an internal one.

Not in scope here

This issue does not advance the parity pin. 22170354d8 is inside the
5559679229..e126687a9a window that #2611
owns.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/vllm/entrypoints/openai/api_server.cpp, then trace the serving calls in serving_chat.cpp and serving_completion.cpp and the body parsing in protocol.cpp. Run the existing socket-level API tests before adding coverage for a valid X-Vllm-Priority header and malformed-header fallback; done means both requests reach the engine with the expected priority.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.