Support for running jobs as kube deployments
- Dominant language
- Java
- Stars
- 163
- Forks
- 65
- Avg merge
- 11h 46m
- Merged PRs (30d)
- 29
Description
**BACKGROUND**
At the moment, all the runs within a `Cloud Pipeline` are launched as kube pods.
This is fine for the batch/ad-hoc jobs, but `Cloud Pipeline` is being used to host certain scientific services, e.g. Shiny server. Those services are then exposed via the [Sharing feature](https://epam.github.io/cloud-pipeline/release_notes/v.0.13/v.0.13_-_Release_notes/#running-instances-sharing-with-other-users-or-groups-of-users) to other users.
Being a pod - such services are totally crashed if something is going wrong within. It would be nice to automatically restart them if a pod is crashed.
**Approach**
* It is proposed to allow users submit a job as a `deployment` (extra option in the API, Web GUI `Launch` form and `pipe` CLI)
* Such jobs shall be scheduled to kube as deployments
* In this case`Autoscaler`/`Pod monitor` shall watch for the deployments, instead of the pods
* As a result - the failed pods will be recreated automatically by the kube itself
* We shall consider `EDGE` service routing changes as well (e.g. recreated pod's container will have a different IP)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.