kserve / kserve/website

docs: add Managed DRA guide for LLMInferenceService

Open
#733 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
113
Forks
193
Avg merge
5d 20h
Merged PRs (30d)
3

Description

## Summary

KServe v0.20 added Managed Dynamic Resource Allocation (DRA) support for LLMInferenceService, allowing dynamic GPU provisioning via annotations. No documentation exists beyond auto-generated CRD API references.

## What needs to be documented

- What DRA is and when to use it vs static GPU `requests/limits`
- Annotation-based configuration: `serving.kserve.io/dra-device-class` and `serving.kserve.io/dra-gpu-count`
- How the controller generates `ResourceClaimTemplates` and injects `ResourceClaims`
- Support for both single-node and multi-node (LeaderWorkerSet) deployments
- Prerequisites: Kubernetes DRA feature gates enabled
- Example YAML

## Related PR

- https://github.com/kserve/kserve/pull/5352

## Since

KServe v0.20

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the related PR #5352 and the existing auto-generated CRD API references for LLMInferenceService. Document DRA prerequisites, annotations, generated claims, single-node and LeaderWorkerSet examples, and the distinction from static GPU requests; the work is done when users can follow a complete example and understand when to use each approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.