vllm-project / vllm-project/agentic-api
Persist reliable Responses continuation checkpoints
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 284
- Forks
- 74
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 93
Description
Parent tracker: #69
Design and measured evidence: #65
Summary
Make Responses and Conversation continuation checkpoints reliable enough to carry certified prefix/artifact metadata without promoting incorrect state.
This is the second milestone, after exact llm-d KV-affine routing is available. Correct routing can operate on ordinary rehydrated full requests; durable token artifacts must not be introduced until checkpoint correctness is defined.
Scope
- Persist immutable execution fingerprint, lineage, renderer-safe boundary proof, token-artifact identity, router-receipt identity, status, and expiration with the accepted response.
- Bind identity to model weights, tokenizer, renderer/template/parser configuration, instructions, tools,
cache_salt, LoRA, multimodal inputs, and format versions. - Fail closed for missing, corrupt, unknown, out-of-order, or partially rehydrated certified history.
- Atomically commit logical output and checkpoint metadata; streaming completion must not imply durable promotion before persistence succeeds.
- Add unique conversation ordering plus compare-and-swap or serialization for concurrent turns.
- Define cancellation, retry, and idempotency behavior so duplicate tool effects or checkpoints cannot be promoted.
- Carry ephemeral prefix/artifact state through gateway-tool subturns and promote only the accepted outer result.
- Measure checkpoint lookup, validation, and persistence overhead separately from vLLM latency.
Acceptance criteria
- Cancellation, retry, concurrent turns, corrupt history, and out-of-order items cannot promote an invalid checkpoint or duplicate tool side effects.
- Rehydration either reconstructs the complete certified model-visible history or returns a typed fallback/failure; it does not silently omit unknown items.
- Logical output and continuation metadata become durable atomically.
- Fingerprint changes invalidate reuse without invalidating the logical conversation fallback.
- Tests cover tool-loop intermediate state, streaming disconnects, persistence failure, and conversation races.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading parent tracker #69 and the design and measured evidence in #65. Trace the checkpoint lifecycle described in the Scope, then use the acceptance criteria to define completion: atomic durable output and metadata, safe rehydration and invalidation, and tests for cancellation, retries, races, corrupt history, tool loops, disconnects, and persistence failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100