vllm-project / vllm-project/agentic-api

Make persisted reasoning replay aware of upstream capabilities

Open
#335 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Rust
Stars
284
Forks
74
Avg merge
1d 17h
Merged PRs (30d)
93

Description

Problem statement / motivation

The gateway preserves reasoning items, but its inference preparation assumes vLLM plaintext reasoning. It rejects encrypted-only reasoning and clears opaque state when plaintext exists. That is intentional for vLLM (#151/#153), yet prevents stateful replay to a compatible upstream that can consume its own opaque reasoning state.

Proposed solution

Separate the public stored reasoning item from a capability-aware inference representation. Retain the existing vLLM policy and add opaque replay only for explicitly compatible upstreams. This is a provider-compatibility follow-up, not a request to decrypt reasoning or translate encrypted state across providers.

Acceptance criteria
  • Reasoning items, summaries, and opaque fields survive public serialization and persistence without exposing opaque payloads in logs.
  • Compatible upstream continuations receive the supported opaque state unchanged and in original item order, including around function calls and outputs.
  • vLLM retains its current plaintext normalization and clear encrypted-only rejection.
  • Model/provider changes cannot silently replay incompatible state; document routing/identity requirements and return a useful error where compatibility cannot be established.
  • Cover HTTP JSON/SSE and WebSocket continuations, stored/manual histories, branching, and compaction boundaries for each supported profile.
  • Add regression tests for the existing vLLM behavior and a pinned compatible-provider reference scenario; document the capability matrix under #314.
Additional context

Related: #151/#153 (completed reasoning retention and safe vLLM replay), #314 (capability contract). The vLLM-first deployment should remain unchanged by default. Cross-provider decryption and general encrypted checkpoint interchange are outside scope.

Sources: model capabilities, preserve reasoning across calls.

Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with crates/agentic-server-core/src/executor/rehydrate.rs and the capability contract described in #314. Trace how stored and manual histories are prepared for HTTP JSON/SSE and WebSocket continuations, including branching and compaction. Done means compatible upstreams preserve opaque state in order, vLLM behavior remains unchanged, incompatible routing returns a useful error, and the required regression tests and capability documentation are added.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.