lablup / lablup/backend.ai

Improve overall user experience of model service

Open
#1,748 0 comments 0 reactions 1 assignee Assigned to @kyujin-cho View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### Main idea

Since the birth of Backend.AI Model Service, main concern of the feature is that it is too hard to utilize for majority of users those who want to serve their own model. To overcome this problem we decided to add several new features on both Core and WebUI, which will potentially enhance overall experience of the Model Service feature.

- Core: New "Dry Run" APIThis new API should validate actual whole lifecycle of the inference session. Its request schema will be identical with the model service creation API. The implementation should first read `model-definition.yml`, create a new inference session accordingly but without the bound routing and endpoint, wait until model server loads, perform a health check (if defined at model definition), and finally terminate the created session. The API should report the whole progress to callee with the help of SSE.
### Alternative ideas

_No response_

### Anything else?

_No response_

```Java
### Tasks
- [ ] https://github.com/lablup/backend.ai/pull/1754
- [ ] Expose `model-definition` validator as a static validation schema (perhaps as a format of JSONSchema)
- [ ] (on WebUI) Implement live `model-definition.yml` editor
- [ ] (on WebUI) Introduce on-the-fly `model-definition.yml` schema validator for the editor
```

JIRA Issue: BA-173

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.