lablup / lablup/backend.ai

More detail Session dependency checks

Open
#1,746 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### Main idea

Currently, Session dependency check has only steps of checking "Sessions.result" column of depending sessions.The value of `result` is `SUCCESS` if the session's main kernel finishes its batch job finish without any error.Else, the `result` would be fail, which means the batch job occurs any error.

We could improve the dependency check logic to control session creation flow with conditions.I suggest to implement like below.

1. Add a condition field to the session dependency (or session) table. Receive it as a string that can be executed as a command in the kernel, and run as a command service in the kernel before the batch compute session ends. The command string parse and validation could be done in manager by adapting minilang. The result of this execution must be a `SUCESS` for the execution to be a success.
1. It would be nice to have an `on_fail` in the condition as well, to cover the case where the execution result is fail (in general, if the batch session occurs any error).
### Reference

- [AWS Sagemaker](https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-steps.html)
- [Azure Pipeline](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=yaml,stages)
### Alternative ideas

_No response_

### Anything else?

_No response_

JIRA Issue: BA-174

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.