vllm-project / vllm-project/aibrix
Inject environment variables into the sidecar for configuring AI Runtime
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
### 🚀 Feature Description and Motivation
Multiple types of information are required during the startup process of AI Runtime:
-Engine side: inference engine type, access address for engine metrics
-Model side: Remote address of the model to be loaded, download authentication information
The above information needs to be injected into the AI Runtime container through environment variables.
### Use Case
_No response_
### Proposed Solution
It is possible to pass the required information in the annotation and inject it into the environment variables of the side car, similar to the implementation of [`Istio metrics merge`](https://github.com/istio/istio/blob/455eb1d46ee25e8dbbffe8792b7a28629548c94e/pkg/kube/inject/webhook.go#L881-L908)
Contributor guide
Assessment
This issue has not been assessed yet.