vllm-project / vllm-project/agentic-api
Support cache-preserving reasoning changes with configuration_update input items
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 289
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Problem statement / motivation
The Responses API can change reasoning effort between responses by appending a configuration_update item while keeping request-level reasoning.effort unchanged. Agentic API has no typed variant for this item. It becomes Unknown, loses its payload, and is excluded from persisted input history.
Proposed solution
Add typed configuration-update items and preserve their position through input handling, storage, rehydration, and upstream serialization. Track the effective effort separately from the initial request-level setting. Gate execution on upstream capability; an unsupported backend must reject the operation before inference instead of silently changing semantics.
Acceptance criteria
- HTTP and WebSocket requests accept supported updates, including manually replayed histories and
previous_response_idcontinuation. - An update applies to subsequent responses until superseded; branching histories preserve their own effective settings.
- The original request-level effort and preceding item history remain unchanged; public response metadata follows the documented contract.
- Reject adjacent updates and unsupported mode combinations. The current reference supports standard single-agent mode, excludes automatic compaction/truncation, and rejects updates at the standalone compact endpoint.
- Explicit
compaction_triggerbehavior is covered, including a fresh update after compaction. - Tests cover malformed updates, unsupported effort, multiple turns, stored/transient state, and unsupported-backend errors. Cache-prefix assertions verify serialized input stability, not an assumed cache hit.
Additional context
Related: #314 (request capabilities), #298 (multi-agent compatibility boundaries). This issue changes API state and forwarding; it does not move prompt rendering or cache execution into the gateway.
Source: change reasoning mid-conversation.
Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with crates/agentic-server-core/src/types/io/input.rs at the reviewed implementation, then trace input handling, storage, rehydration, and upstream serialization from there. Use the acceptance criteria to define done: supported updates preserve history and effective settings across HTTP, WebSocket, continuation, and compaction cases, while malformed or unsupported operations are rejected and the listed state, forwarding, and cache-prefix tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100