vllm-project / vllm-project/agentic-api
Make persisted reasoning replay aware of upstream capabilities
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 284
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Problem statement / motivation
The gateway preserves reasoning items, but its inference preparation assumes vLLM plaintext reasoning. It rejects encrypted-only reasoning and clears opaque state when plaintext exists. That is intentional for vLLM (#151/#153), yet prevents stateful replay to a compatible upstream that can consume its own opaque reasoning state.
Proposed solution
Separate the public stored reasoning item from a capability-aware inference representation. Retain the existing vLLM policy and add opaque replay only for explicitly compatible upstreams. This is a provider-compatibility follow-up, not a request to decrypt reasoning or translate encrypted state across providers.
Acceptance criteria
- Reasoning items, summaries, and opaque fields survive public serialization and persistence without exposing opaque payloads in logs.
- Compatible upstream continuations receive the supported opaque state unchanged and in original item order, including around function calls and outputs.
- vLLM retains its current plaintext normalization and clear encrypted-only rejection.
- Model/provider changes cannot silently replay incompatible state; document routing/identity requirements and return a useful error where compatibility cannot be established.
- Cover HTTP JSON/SSE and WebSocket continuations, stored/manual histories, branching, and compaction boundaries for each supported profile.
- Add regression tests for the existing vLLM behavior and a pinned compatible-provider reference scenario; document the capability matrix under #314.
Additional context
Related: #151/#153 (completed reasoning retention and safe vLLM replay), #314 (capability contract). The vLLM-first deployment should remain unchanged by default. Cross-provider decryption and general encrypted checkpoint interchange are outside scope.
Sources: model capabilities, preserve reasoning across calls.
Reviewed at cdac36f: implementation. Documentation checked September 18, 2026; pin the implementation-time API/model version when recording conformance.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with crates/agentic-server-core/src/executor/rehydrate.rs and the capability contract described in #314. Trace how stored and manual histories are prepared for HTTP JSON/SSE and WebSocket continuations, including branching and compaction. Done means compatible upstreams preserve opaque state in order, vLLM behavior remains unchanged, incompatible routing returns a useful error, and the required regression tests and capability documentation are added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100