docs: document model name-based routing for LLMInferenceService
- Dominant language
- MDX
- Stars
- 113
- Forks
- 193
- Avg merge
- 5d 20h
- Merged PRs (30d)
- 3
Description
## Summary
KServe v0.19 added model name-based routing for LLMInferenceService, enabling shared-gateway deployments. While the Envoy AI Gateway integration documents `x-ai-eg-model` header routing, the generic `X-Gateway-Model-Name` header routing is not documented.
## What needs to be documented
- How model name-based routing works with the `X-Gateway-Model-Name` header
- Configurable header name
- How LoRA adapter requests are automatically routed through the same backend by dynamically expanding header matches
- Shared-gateway deployment pattern where a single gateway dispatches to multiple models
- Integration with components like llm-d BBR (IPP) or gateway-specific native features
- `-model-routing` suffix address types in status
## Related PR
- https://github.com/kserve/kserve/pull/5521
## Since
KServe v0.19
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the related PR #5521 and the existing Envoy AI Gateway integration documentation. Document generic X-Gateway-Model-Name routing, configurable header names, LoRA adapter routing, shared-gateway deployments, llm-d BBR or gateway-native integrations, and -model-routing status addresses; done means each listed behavior is covered.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100