futureverse / futureverse/future.apply

Graceful error handling in future_lapply

Open
#32 6 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
R
Stars
218
Forks
20
Avg merge
2d 9h
Merged PRs (30d)
1

Description

I've been using `future_lapply` to submit jobs to a cluster of gcloud compute nodes - which is awesome!! Thanks so much for adding this user-friendly function.

One issue I'm running into, however, concerns the behavior when one of my gcloud compute nodes goes down unexpectedly.

Here's what I see:

```
Error in unserialize(node$con) :
Failed to retrieve the value of ClusterFuture from cluster node #1 (on ‘35.190.151.35’). The reason reported was ‘error reading from connection’
Calls: future_lapply ... FutureRegistry -> collectValues -> value -> value.ClusterFuture
Execution halted
```

Is there a good way to catch these errors for individual nodes, so that I can still access results from the nodes which are still executing? IE, can we pass an `on.error` function to call to `values(fs)`?

For now I have wrapped these in a call to `withCallingHandlers()` to catch & ignore this error. I haven't tested it in a real-life situation, but assuming it works this may be something to include in your vignette [Common Issues with Solutions](https://cran.r-project.org/web/packages/future/vignettes/future-2-issues.html).

The code currently looks like something the following:

```
withCallingHandlers({
remote_jobs <- future_lapply(seq_len(num_data_sims),
execute_simtest,
future.seed = 0xBEEF,
future.scheduling = 2
)
}, FutureError = function(e) NULL
)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.