vllm-project / vllm-project/aibrix

Inject environment variables into the sidecar for configuring AI Runtime

Open
#166 2 comments 0 reactions 1 assignee Claimed by @brosoul View on GitHub
area/runtime kind/feature priority/critical-urgent
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

### 🚀 Feature Description and Motivation

Multiple types of information are required during the startup process of AI Runtime:
-Engine side: inference engine type, access address for engine metrics
-Model side: Remote address of the model to be loaded, download authentication information

The above information needs to be injected into the AI Runtime container through environment variables.

### Use Case

_No response_

### Proposed Solution

It is possible to pass the required information in the annotation and inject it into the environment variables of the side car, similar to the implementation of [`Istio metrics merge`](https://github.com/istio/istio/blob/455eb1d46ee25e8dbbffe8792b7a28629548c94e/pkg/kube/inject/webhook.go#L881-L908)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.