vllm-project / vllm-project/agentic-api

Preserve request capabilities and transport metadata across execution paths

Open
#314 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement enterprise-readiness
Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Problem statement / motivation

Enterprise clients need stable request semantics when a request selects pass-through, stateful execution, or built-in tool execution. Provide an explicit capability contract so required fields and metadata are preserved, deliberately transformed, or rejected with a useful error.

Proposed solution
  • Document and test field handling for Responses and Messages, including reasoning, structured output, caching, metadata, multimodal content, tool declarations, and supported extensions.
  • Add typed support for required executor fields; retain the existing raw-body preservation boundary for native Messages. Do not round-trip native requests through a partial type or introduce untyped public core APIs.
  • Define which request headers/query parameters and response metadata cross each transport boundary, including version/beta controls, tracing, request IDs, rate limits, and retry guidance.
  • Separate client authentication, trusted proxy identity, inference credentials, and tool credentials. Filter hop-by-hop headers and invalidate representation headers after transformations.
Acceptance criteria
  • A published per-path matrix states supported, transformed, and rejected capabilities, with explicit extension handling.
  • Required request semantics do not silently disappear when enabling storage or built-in tools; unsupported behavior returns a protocol-appropriate validation error before inference/tool effects.
  • Tests compare equivalent pass-through and executor-backed requests over HTTP JSON/SSE and Responses WebSocket where supported.
  • Native Messages tests preserve system/content blocks, thinking signatures, cache controls, tool_result errors, version/beta controls, and supported future fields.
  • Tests cover safe metadata propagation, credential separation, successful and failed requests, and existing error-header behavior from #250.
  • Standard conversation references and creation payloads are covered through #155; that issue remains the owner of Conversations resource implementation.
Additional context

Enterprise Readiness enhancement. Coordinate with #107 for trusted identity and #155 for conversation compatibility. Token counting must use the declared model/backend capability; do not substitute generation for a count_tokens operation.

Source review at afa6c14: Responses request types, Messages request context, and proxy metadata handling. This is a capability enhancement, not a claim of complete upstream API conformance.

Enterprise Readiness tracker: https://github.com/vllm-project/agentic-api/issues/316

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/agentic-server-core/src/types/request_response.rs, executor/messages_context.rs, and proxy.rs at source review afa6c14. Map the stated request paths, transport boundaries, headers, metadata, credentials, and native Messages fields against the acceptance criteria, coordinating with #107 and #155. Done requires the published capability matrix, validation behavior, and comprehensive HTTP JSON/SSE and WebSocket tests described here.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend-api-design, security, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.