kserve / kserve/website

docs: document graceful pod shutdown (preStop hook and vLLM shutdown-timeout)

Open
#726 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
113
Forks
193
Avg merge
5d 20h
Merged PRs (30d)
3

Description

## Summary

KServe v0.19 implemented graceful drain for LLM inference pods to prevent in-flight request drops during rolling updates or scale-down events. This involves a preStop hook and vLLM `--shutdown-timeout` flag. There is no documentation for this feature.

## What needs to be documented

- The drain sequence: `preStop sleep (15s) → EPP stops routing → SIGTERM → vLLM drains (≤45s) → pod exits`
- preStop hook (`/bin/sleep 15`) purpose: gives EPP time to stop routing new requests
- vLLM `--shutdown-timeout` flag: computed as `terminationGracePeriodSeconds - 15`, defaults to 45s
- `terminationGracePeriodSeconds` increased from 30s to 60s across all vLLM templates
- How users can override via `spec.template.terminationGracePeriodSeconds`

## Related PRs

- https://github.com/kserve/kserve/pull/5485 (preStop hook + terminationGracePeriod)
- https://github.com/kserve/kserve/pull/5496 (vLLM shutdown-timeout)

## Since

KServe v0.19

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing related PRs 5485 and 5496 and the KServe v0.19 vLLM documentation entry points. Document the complete drain sequence, the preStop and shutdown-timeout behavior, the 60-second termination grace default, and the spec.template override so users can verify configuration and expected shutdown timing.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.