vllm-project / vllm-project/aibrix

[Discussion][Umbrella] ModelAdapter Issues

Open
#700 1 comment 1 reaction 0 assignees View on GitHub
area/lora kind/enhancement priority/critical-urgent
Dominant language
Go
Stars
5.1k
Forks
694
Avg merge
1d 20h
Merged PRs (30d)
98

Description

Track some issues I have for ModelAdapter, some are big concerns, some are small, just for discussions so I can help with the project.
Will update them based on the latest design.

- [ ] 1. `modelAdapterStatus.Instances` will record all the Pod names as a list, will that lead to a explosion of the list length if we have lots of Pods
- [ ] 2. we'll load/unload the adapter in the runtime of reconciliation via http requests, which might be too heavy for the controller, especially when thousands of adapters reconciling the same time, maybe agent?
- [x] 3. we'll validate the CRD in the controller, which should be delegated to the webhook, see separate issue: https://github.com/aibrix/aibrix/issues/710
- [ ] 4. we have a scheduler in the controller, which usually should be a separate component, but I think it's ok as a start
- [ ] 5. TODO: when removing modelAdapter, we only unload 0-index instance rather than the whole list

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.