NatLabRockies / NatLabRockies/gaps
Batch randomly cancels in-progress jobs if re-submitting
@ppinchuk is already working on this.
Since Feb 13, 2025.
- Dominant language
- Python
- Stars
- 1
- Forks
- 10
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 8
Description
Sometimes kicking off a batch job while it is already running cancels the set that was kicked off first (but does not cancel all sets that are running).
You submit the batch, then change the configs and kick off the batch again - when you do that, only one of the two sets that were running get canceled. However, you wouldn't expect anything to be cancelled because it was "submitted"/running.
No config paths were changed.
Use case for this would be to kick batch off again for some failed jobs while others are still running.
This is all using --monitor-background.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.