futureverse / futureverse/progressr

Progress bar only on sequential and multicore, missing on multisession and cluster

Open
#191 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
299
Forks
11
PR merge metrics
No merged PRs in 30d

Description

I have incorporated `progressr` progressor into a fairly complex R package that uses `%dofuture%` to split large objects across workers.

The progress bar shows up when called under `plan(sequential)` or, in Linux, under `plan(multicore)`, but fails to show up with either `multisession` or `cluster`. This happens both in Linux and Windows, the later tested on a clean R 4.5.2 installation. After setting `multisession`, `handlers('debug')` also fails to print any messages.

Running the examples in the 'progressr' package works without problem, so there might be something in our packages that is affecting the progressor. But the fact of only failing under some plans puzzles me.

I have placed example code under [this gist](https://gist.github.com/iagomosqueira/d4783c475f201a26ec1332d32f0fc914). I am afraid it requires installation of quite a few packages.

On multicore we are getting warnings about some progression updates not being listened to, but running with 'debug' did not clarify what is wrong

```
> plan(multicore, workers=2)
> tes <- mp(om, oem=oem, ctrl=control, args=list(iy=2021, fy=2026))
Running on 2 nodes.
[08:51:13.790] (0.000s => +0.014s) initiate: 0/6 (+0) '' {clear=TRUE, enabled=TRUE, status=}
[08:51:13.791] (0.001s => +0.013s) update: 1/6 (+1) 'year: 2021' {clear=TRUE, enabled=TRUE, status=}
[08:51:14.776] (0.986s => +0.017s) update: 2/6 (+1) 'year: 2022' {clear=TRUE, enabled=TRUE, status=}
[08:51:15.728] (1.939s => +0.016s) update: 3/6 (+1) 'year: 2023' {clear=TRUE, enabled=TRUE, status=}
[08:51:18.644] (4.854s => +0.004s) update: 4/6 (+1) 'year: 2024' {clear=TRUE, enabled=TRUE, status=}
[08:51:19.646] (5.856s => +0.013s) update: 5/6 (+1) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
[08:51:20.696] (6.906s => +0.001s) shutdown: 6/6 (+1) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
[08:51:20.697] (6.907s => +0.002s) shutdown: 6/6 (+0) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
Warning: The global progression handlers received a progression ‘update’ request (amount=1; msg=‘year: 2024’), but is no longer listening to this progressor. This can happen when code signals more progress updates than steps in the progressor. To troubleshoot this, retry with progressr::handlers("debug")
Warning: The global progression handlers received a progression ‘update’ request (amount=1; msg=‘year: 2025’), but is no longer listening to this progressor. This can happen when code signals more progress updates than steps in the progressor. To troubleshoot this, retry with progressr::handlers("debug")
```

It does not happen with 'sequential'

```
> plan(sequential)
tes <- mp(om, oem=oem, ctrl=control, args=list(iy=2021, fy=2026))
TRACKER: .Random.seed changed
> tes <- mp(om, oem=oem, ctrl=control, args=list(iy=2021, fy=2026))
[08:50:46.784] (0.000s => +0.013s) initiate: 0/6 (+0) '' {clear=TRUE, enabled=TRUE, status=}
[08:50:46.850] (0.066s => +0.065s) update: 1/6 (+1) 'year: 2021' {clear=TRUE, enabled=TRUE, status=}
[08:50:51.621] (4.837s => +0.001s) update: 2/6 (+1) 'year: 2022' {clear=TRUE, enabled=TRUE, status=}
[08:50:53.994] (7.210s => +0.001s) update: 3/6 (+1) 'year: 2023' {clear=TRUE, enabled=TRUE, status=}
[08:50:55.918] (9.133s => +0.001s) update: 4/6 (+1) 'year: 2024' {clear=TRUE, enabled=TRUE, status=}
[08:50:58.667] (11.883s => +0.000s) update: 5/6 (+1) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
[08:51:01.272] (14.487s => +0.000s) shutdown: 6/6 (+1) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
[08:51:01.272] (14.488s => +0.001s) shutdown: 6/6 (+0) 'year: 2025' {clear=TRUE, enabled=TRUE, status=}
```

Any suggestions on what to check?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.