Enhance usability of runtime variant feature
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Motivation
Model service’s Runtime Variant feature is a way to eliminate process of writing down model definition file when launching the model service. Currently, to launch the model service with the runtime variant feature enabled, the user first have to load the supported image to the [Backend.AI](http://Backend.AI), then start the model service with both valid image and runtime variant option selected. This process misleads most of the user without the context - they understand the dropdown lets them gain access to the selected inference framework even on unsupported images.
{width=671 height=222}
## Required Features
- Update [Backend.AI](http://Backend.AI) Image label specification to explicitly advertise which runtime variant the image supports
- Revamp both Manager API and Web UI to restrict user’s runtime variant selection
- But we still need user to allow making use of image’s arbitrary `CMD` field to start the inference framework
- Initial idea is to remove all other variants from the dropdown and leave only `Custom (Default)` and `Predefined Image Command`, and create a new text area (which is hidden by default) to expose the selected image’s supported runtime variant
- Add new `runtime_variant` field to image GQL
- Ideation - will a single image support more than one runtime variants?
## Impact
- [Backend.AI](http://Backend.AI)’s service creation (`POST /service`) API will be updated
- [Backend.AI](http://Backend.AI) WebUI will be updated
## Testing Scenarios
WIP
JIRA Issue: BA-628
Contributor guide
Assessment
This issue has not been assessed yet.