docs: document graceful pod shutdown (preStop hook and vLLM shutdown-timeout)
- Dominant language
- MDX
- Stars
- 113
- Forks
- 193
- Avg merge
- 5d 20h
- Merged PRs (30d)
- 3
Description
## Summary
KServe v0.19 implemented graceful drain for LLM inference pods to prevent in-flight request drops during rolling updates or scale-down events. This involves a preStop hook and vLLM `--shutdown-timeout` flag. There is no documentation for this feature.
## What needs to be documented
- The drain sequence: `preStop sleep (15s) → EPP stops routing → SIGTERM → vLLM drains (≤45s) → pod exits`
- preStop hook (`/bin/sleep 15`) purpose: gives EPP time to stop routing new requests
- vLLM `--shutdown-timeout` flag: computed as `terminationGracePeriodSeconds - 15`, defaults to 45s
- `terminationGracePeriodSeconds` increased from 30s to 60s across all vLLM templates
- How users can override via `spec.template.terminationGracePeriodSeconds`
## Related PRs
- https://github.com/kserve/kserve/pull/5485 (preStop hook + terminationGracePeriod)
- https://github.com/kserve/kserve/pull/5496 (vLLM shutdown-timeout)
## Since
KServe v0.19
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing related PRs 5485 and 5496 and the KServe v0.19 vLLM documentation entry points. Document the complete drain sequence, the preStop and shutdown-timeout behavior, the 60-second termination grace default, and the spec.template override so users can verify configuration and expected shutdown timing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100