vllm-project / vllm-project/agentic-api
Preserve request capabilities and transport metadata across execution paths
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 284
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Problem statement / motivation
Enterprise clients need stable request semantics when a request selects pass-through, stateful execution, or built-in tool execution. Provide an explicit capability contract so required fields and metadata are preserved, deliberately transformed, or rejected with a useful error.
Proposed solution
- Document and test field handling for Responses and Messages, including reasoning, structured output, caching, metadata, multimodal content, tool declarations, and supported extensions.
- Add typed support for required executor fields; retain the existing raw-body preservation boundary for native Messages. Do not round-trip native requests through a partial type or introduce untyped public core APIs.
- Define which request headers/query parameters and response metadata cross each transport boundary, including version/beta controls, tracing, request IDs, rate limits, and retry guidance.
- Separate client authentication, trusted proxy identity, inference credentials, and tool credentials. Filter hop-by-hop headers and invalidate representation headers after transformations.
Acceptance criteria
- A published per-path matrix states supported, transformed, and rejected capabilities, with explicit extension handling.
- Required request semantics do not silently disappear when enabling storage or built-in tools; unsupported behavior returns a protocol-appropriate validation error before inference/tool effects.
- Tests compare equivalent pass-through and executor-backed requests over HTTP JSON/SSE and Responses WebSocket where supported.
- Native Messages tests preserve system/content blocks, thinking signatures, cache controls, tool_result errors, version/beta controls, and supported future fields.
- Tests cover safe metadata propagation, credential separation, successful and failed requests, and existing error-header behavior from #250.
- Standard conversation references and creation payloads are covered through #155; that issue remains the owner of Conversations resource implementation.
Additional context
Enterprise Readiness enhancement. Coordinate with #107 for trusted identity and #155 for conversation compatibility. Token counting must use the declared model/backend capability; do not substitute generation for a count_tokens operation.
Source review at afa6c14: Responses request types, Messages request context, and proxy metadata handling. This is a capability enhancement, not a claim of complete upstream API conformance.
Enterprise Readiness tracker: https://github.com/vllm-project/agentic-api/issues/316
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with crates/agentic-server-core/src/types/request_response.rs, executor/messages_context.rs, and proxy.rs at source review afa6c14. Map the stated request paths, transport boundaries, headers, metadata, credentials, and native Messages fields against the acceptance criteria, coordinating with #107 and #155. Done requires the published capability matrix, validation behavior, and comprehensive HTTP JSON/SSE and WebSocket tests described here.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend-api-design, security, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100