futureverse / futureverse/future.batchtools

LSF/OpenLava template: Signal SIGINT by default to allow R to catch it gracefully

Open Beginner friendly
#105 0 comments 0 reactions 0 assignees View on GitHub
feature/resources scheduler/lsf
Dominant language
R
Stars
87
Forks
10
PR merge metrics
No merged PRs in 30d

Description

When a job is approaching it's maximum run-time limit, LSF sends a `SIGINT` 10 seconds before sending a `SIGKILL` (abrupt; not capturable).

R code can handle the `SIGINT`, but the 10-second heads up might be a bit tight. To adjust this, we can use LSF options `-wa ` and `-wt `, e.g. `-wa INT -wt 1` will give us a 60-second heads up.

# Idea

Update the default LSF template to signal `SIGINT` 60 seconds in advance. Something like:

```bash
<%
## Signal SIGINT 60 seconds (1 minute) before the run-time limit
if (is.null(resources[["wa"]])) {
resources[["wa"]] <- "INT"
}
if (is.null(resources[["wt"]])) {
resources[["wt"]] <- 1
}
%>
```

Contributor guide

Open the contributing guide

Research direction

Start by locating the default LSF/OpenLava template and its handling of the wa and wt resources. Verify that the defaults request SIGINT 1 minute before the run-time limit while preserving user-provided values; done when the generated submission uses those defaults and existing overrides still work.

Written by the indexing model from the issue text.

Assessment

Tech stack
hpc, r
Domain
distributed-systems
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.