kserve / kserve/website

docs: add LocalModelCache support for LLMInferenceService

Open Beginner friendly
#727 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
113
Forks
193
Avg merge
5d 20h
Merged PRs (30d)
3

Description

## Summary

KServe v0.19 added LocalModelCache support for LLMInferenceService, but the existing LocalModelCache documentation (`docs/model-serving/generative-inference/modelcache/localmodel.md`) only covers usage with InferenceService.

## What needs to be documented

- How LocalModelCache integrates with LLMInferenceService
- Webhook defaulter behavior: automatic label/annotation setting when model URI matches a cache
- URI rewriting from `hf://model` to `pvc:///models//`
- Delete protection: cache deletion is blocked when active LLMInferenceServices reference it
- Label propagation to Deployment and LeaderWorkerSet pod templates
- Example YAML for LLMInferenceService using LocalModelCache and LocalModelNamespaceCache

## Related PR

- https://github.com/kserve/kserve/pull/5318

## Since

KServe v0.19

Contributor guide

No contributing guide indexed for this repository

Research direction

Update docs/model-serving/generative-inference/modelcache/localmodel.md, using the related PR #5318 as context for LocalModelCache support with LLMInferenceService. Document webhook defaults, URI rewriting, delete protection, label propagation, and YAML examples for LocalModelService and LocalModelNamespaceCache. Done means the existing documentation covers each listed behavior and example.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.