vllm-project / vllm-project/aibrix
[Discussion][Umbrella] ModelAdapter Issues
- Dominant language
- Go
- Stars
- 5.1k
- Forks
- 694
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 98
Description
Track some issues I have for ModelAdapter, some are big concerns, some are small, just for discussions so I can help with the project.
Will update them based on the latest design.
- [ ] 1. `modelAdapterStatus.Instances` will record all the Pod names as a list, will that lead to a explosion of the list length if we have lots of Pods
- [ ] 2. we'll load/unload the adapter in the runtime of reconciliation via http requests, which might be too heavy for the controller, especially when thousands of adapters reconciling the same time, maybe agent?
- [x] 3. we'll validate the CRD in the controller, which should be delegated to the webhook, see separate issue: https://github.com/aibrix/aibrix/issues/710
- [ ] 4. we have a scheduler in the controller, which usually should be a separate component, but I think it's ok as a start
- [ ] 5. TODO: when removing modelAdapter, we only unload 0-index instance rather than the whole list
Contributor guide
Assessment
This issue has not been assessed yet.