mudler / mudler/vllm.cpp

feat(SERVE-COMPLETION-LONGTAIL): session_id is not carried from the request to the engine, and no handler reads headers

Open
#2,658 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
423
Forks
53
Avg merge
20h 26m
Merged PRs (30d)
310

Description

Row: SERVE-COMPLETION-LONGTAIL

Found by wave PORTQ-2 re-deriving PORT-NOW entry 60 of 5559679229..e126687a9a
(#2646). Upstream:
f57123aa2d vllm#48048, "feat(frontend): session id plumbing into requests".
Previously read by #2524's sample (.agents/sync/2026-09-01-cdefd9d.md §13 item
9, "API-only"); re-verified at e24ec8bfd, still holds.

Absent, checked twice and in this tree's own vocabulary

session_id, SessionId, X-Session-ID and session-id return zero lines
across include/, src/ and tests/. Searching the CONCEPT rather than the
upstream spelling gives the same answer: the request carries only request_id
and priority (include/vllm/v1/engine/types.h:77,88, :176), and the server
reads no request headers at all —
src/vllm/entrypoints/openai/api_server.cpp:1176-1179 binds
[this, write](const httplib::Request& req, httplib::Response& res) { write(handle_chat_completions(req.body), res); },
discarding the header-bearing object. The only get_header_value in the tree is
in the HuggingFace HTTP client (src/vllm/transformers_utils/downloader.cpp:65).

Unlike most of this tranche, every surface the commit touches is ported here
(the OpenAI request structs, serving_chat / serving_completion,
EngineCoreRequest, v1::Request), which is why this is portable work rather
than an absent surface.

What the port needs

  1. An optional session_id on ChatCompletionRequest / CompletionRequest in
    include/vllm/entrypoints/openai/protocol.h and its parse.
  2. The resolution order: body field, then the X-Session-ID header, then
    vllm_xargs["session_id"]. Our vllm_xargs is itself deferred
    (protocol.h:37), so only the first two arms are reachable and the third is
    owed rather than ported.
  3. A header-plumbing seam. api_server.cpp:1176-1179 drops
    httplib::Request today, so the handler signature has to carry headers. This
    is the largest single piece and it does NOT exist upstream, where
    raw_request is already threaded.
  4. std::optional<std::string> session_id on EngineCoreRequest
    (include/vllm/v1/engine/types.h) and v1::Request
    (include/vllm/v1/request.h), copied alongside src/vllm/v1/request.cpp:109.
  5. A reader, or the field lands dead under AGENTS.md §"Nothing lands dead".
    LogRequestReceived (include/vllm/entrypoints/openai/request_logger.h:32-35)
    is the natural one and is what makes the field reachable from the server's
    default configuration.

Roughly 150-220 lines over ~10 files plus one test; the header seam is the part
to size carefully.

Open question

Whether upstream has any CONSUMER of session_id beyond carrying it. At this
commit it is pure plumbing. If it is carry-only upstream too, our port must add a
reader anyway to be reachable — settled by grepping session_id in vLLM at a
later revision for a read site.

Nothing was executed for this finding.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the request types in include/vllm/entrypoints/openai/protocol.h and the handler in src/vllm/entrypoints/openai/api_server.cpp, then trace propagation through include/vllm/v1/engine/types.h, include/vllm/v1/request.h, and src/vllm/v1/request.cpp. Add the documented body/header plumbing and a reader in include/vllm/entrypoints/openai/request_logger.h; done includes the one planned test passing and no dead session_id field.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.