vllm-project / vllm-project/agentic-api
Add Responses WebSocket mid-turn steering and automatic continuation
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 284
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Problem statement / motivation
Agentic API currently accepts only response.create WebSocket events. Clients cannot send a correction while work is running using the documented response.steer contract. Concurrent stream_id lanes and multi-agent injection are different capabilities.
Proposed solution
Add typed steering commands and response events. Route them to the active response owner, queue accepted user input, finish the current output item and already-running built-in tool work, and create a continuation with the update. Preserve completed work and do not re-execute earlier actions.
Acceptance criteria
- Validate
response.steerfields and supported user input against the pinned reference contract, including response identity and same-connection scope. - Implement accepted, pending, and failed steering events with stable steer IDs and defined ordering.
- An interrupted response reports
response.incompletewith reasonsteered; a response that finishes first can still produce the queued continuation. - Automatic continuations inherit settings and apply token/tool-call limits per response.
- Pending client tool outputs or approvals delay steering correctly; a matching
response.createconsumes queued input once without requiring its resubmission. - Tests cover repeated steering, completion races, malformed input, pending calls, multiplexed-lane isolation, backpressure, disconnect, and cancellation. Connection-local steering must not be restored as durable accepted work after reconnect.
- Unsupported backend/mode combinations fail explicitly. Record a duplex OpenAI-reference exchange and replay the equivalent gateway scenario.
Additional context
Related: #239 (completed WebSocket multiplexing), #300 (multi-agent injection), #145 (MCP approval), #241/#244 (streaming boundaries). Reuse the request coordinator and ordered relay. Transport decodes commands; orchestration owns continuation; shared synchronous ingestion owns item assembly. Queues require entry/byte bounds and defined shutdown behavior.
Source: mid-turn steering.
Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with crates/agentic-server/src/handler/websocket/responses.rs, then trace the request coordinator, ordered relay, and shared synchronous ingestion entry points named in the issue. Compare the pinned steering reference and record a duplex OpenAI-reference exchange before implementing. Done means all listed steering, continuation, race, isolation, queue, disconnect, cancellation, and unsupported-mode tests pass, including replay of the equivalent gateway scenario.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100