futureverse / futureverse/future.batchtools
batchtools_slurm futures block (Was: stdout/stderr to log file only when using slurm?)
- Lingua principale
- R
- Stelle
- 87
- Fork
- 10
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
I have a loop like this:
```r
for (i in 14:11) {
f <- future({
warning("Sleeping ", i, " seconds")
Sys.sleep(i)
saveRDS(i^2, paste0("tmp-data/res", i))
})
}
```
and I would to output the warning in the log files only.
I've seen the new option `split` in a blog post to do both (but I didn't find it in the documentation).
The problem is that capturing this output and returning it to the main session usually blocks my session for a long time (sometimes minutes), and I would like to avoid this.
Basically, is there any way to use future/future.batchtools just to submit jobs on a slurm cluster without returning anything to the session sending them?
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start by reproducing the shown future() loop with future.batchtools on a Slurm cluster and investigate the mentioned split option for output handling. Done means jobs can be submitted without the sending session waiting for captured output, while warnings are written only to the Slurm log files.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- r
- Ambito
- hpc
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100