futureverse / futureverse/future.apply
Graceful error handling in future_lapply
- Dominant language
- R
- Stars
- 218
- Forks
- 20
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 1
Description
I've been using `future_lapply` to submit jobs to a cluster of gcloud compute nodes - which is awesome!! Thanks so much for adding this user-friendly function.
One issue I'm running into, however, concerns the behavior when one of my gcloud compute nodes goes down unexpectedly.
Here's what I see:
```
Error in unserialize(node$con) :
Failed to retrieve the value of ClusterFuture from cluster node #1 (on ‘35.190.151.35’). The reason reported was ‘error reading from connection’
Calls: future_lapply ... FutureRegistry -> collectValues -> value -> value.ClusterFuture
Execution halted
```
Is there a good way to catch these errors for individual nodes, so that I can still access results from the nodes which are still executing? IE, can we pass an `on.error` function to call to `values(fs)`?
For now I have wrapped these in a call to `withCallingHandlers()` to catch & ignore this error. I haven't tested it in a real-life situation, but assuming it works this may be something to include in your vignette [Common Issues with Solutions](https://cran.r-project.org/web/packages/future/vignettes/future-2-issues.html).
The code currently looks like something the following:
```
withCallingHandlers({
remote_jobs <- future_lapply(seq_len(num_data_sims),
execute_simtest,
future.seed = 0xBEEF,
future.scheduling = 2
)
}, FutureError = function(e) NULL
)
```
Contributor guide
Assessment
This issue has not been assessed yet.