iterative / iterative/cml

CML kubernetes pod destruction - job still running

Open
#1,332 5 comments 0 reactions 1 assignee Claimed by @0x2b3bfa0 View on GitHub
external-request invalid p2-nice-to-have
Dominant language
JavaScript
Stars
4.2k
Forks
345
PR merge metrics
No merged PRs in 30d

Description

Hi this problem occurred after https://github.com/iterative/cml/issues/1330 was fixed. Somehow the runner is still running when the termination request is made. Can/should I set a "grace period" somewhere?

```
{"level":"info","message":"Unregistering runner cml-cml-runner-1x1f33ub-5fg9nhw1..."}
{"level":"info","message":"GET /repos/cml-test/cml-tutorial-1/actions/runners?per_page=100 - 200 in 105ms"}
{"level":"info","message":"DELETE /repos/cml-test/cml-tutorial-1/actions/runners/30 - 422 in 136ms"}
{"level":"warn","message":"\tCancelling shutdown: Bad request - Runner \"cml-cml-runner-1x1f33ub-5fg9nhw1\" is still running a job\""}
```
EDIT:
Found a similar issue https://github.com/iterative/cml/issues/1103.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.