vllm-project / vllm-project/agentic-api

Support cache-preserving reasoning changes with configuration_update input items

Open
#331 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
289
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Problem statement / motivation

The Responses API can change reasoning effort between responses by appending a configuration_update item while keeping request-level reasoning.effort unchanged. Agentic API has no typed variant for this item. It becomes Unknown, loses its payload, and is excluded from persisted input history.

Proposed solution

Add typed configuration-update items and preserve their position through input handling, storage, rehydration, and upstream serialization. Track the effective effort separately from the initial request-level setting. Gate execution on upstream capability; an unsupported backend must reject the operation before inference instead of silently changing semantics.

Acceptance criteria
  • HTTP and WebSocket requests accept supported updates, including manually replayed histories and previous_response_id continuation.
  • An update applies to subsequent responses until superseded; branching histories preserve their own effective settings.
  • The original request-level effort and preceding item history remain unchanged; public response metadata follows the documented contract.
  • Reject adjacent updates and unsupported mode combinations. The current reference supports standard single-agent mode, excludes automatic compaction/truncation, and rejects updates at the standalone compact endpoint.
  • Explicit compaction_trigger behavior is covered, including a fresh update after compaction.
  • Tests cover malformed updates, unsupported effort, multiple turns, stored/transient state, and unsupported-backend errors. Cache-prefix assertions verify serialized input stability, not an assumed cache hit.
Additional context

Related: #314 (request capabilities), #298 (multi-agent compatibility boundaries). This issue changes API state and forwarding; it does not move prompt rendering or cache execution into the gateway.

Source: change reasoning mid-conversation.

Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/agentic-server-core/src/types/io/input.rs at the reviewed implementation, then trace input handling, storage, rehydration, and upstream serialization from there. Use the acceptance criteria to define done: supported updates preserve history and effective settings across HTTP, WebSocket, continuation, and compaction cases, while malformed or unsupported operations are rejected and the listed state, forwarding, and cache-prefix tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.