vllm-project / vllm-project/agentic-api

Add Responses WebSocket mid-turn steering and automatic continuation

Open
#333 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Problem statement / motivation

Agentic API currently accepts only response.create WebSocket events. Clients cannot send a correction while work is running using the documented response.steer contract. Concurrent stream_id lanes and multi-agent injection are different capabilities.

Proposed solution

Add typed steering commands and response events. Route them to the active response owner, queue accepted user input, finish the current output item and already-running built-in tool work, and create a continuation with the update. Preserve completed work and do not re-execute earlier actions.

Acceptance criteria
  • Validate response.steer fields and supported user input against the pinned reference contract, including response identity and same-connection scope.
  • Implement accepted, pending, and failed steering events with stable steer IDs and defined ordering.
  • An interrupted response reports response.incomplete with reason steered; a response that finishes first can still produce the queued continuation.
  • Automatic continuations inherit settings and apply token/tool-call limits per response.
  • Pending client tool outputs or approvals delay steering correctly; a matching response.create consumes queued input once without requiring its resubmission.
  • Tests cover repeated steering, completion races, malformed input, pending calls, multiplexed-lane isolation, backpressure, disconnect, and cancellation. Connection-local steering must not be restored as durable accepted work after reconnect.
  • Unsupported backend/mode combinations fail explicitly. Record a duplex OpenAI-reference exchange and replay the equivalent gateway scenario.
Additional context

Related: #239 (completed WebSocket multiplexing), #300 (multi-agent injection), #145 (MCP approval), #241/#244 (streaming boundaries). Reuse the request coordinator and ordered relay. Transport decodes commands; orchestration owns continuation; shared synchronous ingestion owns item assembly. Queues require entry/byte bounds and defined shutdown behavior.

Source: mid-turn steering.

Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/agentic-server/src/handler/websocket/responses.rs, then trace the request coordinator, ordered relay, and shared synchronous ingestion entry points named in the issue. Compare the pinned steering reference and record a duplex OpenAI-reference exchange before implementing. Done means all listed steering, continuation, race, isolation, queue, disconnect, cancellation, and unsupported-mode tests pass, including replay of the equivalent gateway scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.