Improve overall user experience of model service
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
### Main idea
Since the birth of Backend.AI Model Service, main concern of the feature is that it is too hard to utilize for majority of users those who want to serve their own model. To overcome this problem we decided to add several new features on both Core and WebUI, which will potentially enhance overall experience of the Model Service feature.
- Core: New "Dry Run" APIThis new API should validate actual whole lifecycle of the inference session. Its request schema will be identical with the model service creation API. The implementation should first read `model-definition.yml`, create a new inference session accordingly but without the bound routing and endpoint, wait until model server loads, perform a health check (if defined at model definition), and finally terminate the created session. The API should report the whole progress to callee with the help of SSE.
### Alternative ideas
_No response_
### Anything else?
_No response_
```Java
### Tasks
- [ ] https://github.com/lablup/backend.ai/pull/1754
- [ ] Expose `model-definition` validator as a static validation schema (perhaps as a format of JSONSchema)
- [ ] (on WebUI) Implement live `model-definition.yml` editor
- [ ] (on WebUI) Introduce on-the-fly `model-definition.yml` schema validator for the editor
```
JIRA Issue: BA-173
Contributor guide
Assessment
This issue has not been assessed yet.