Reduce number of .yml reads in loop_tasks
- Dominant language
- R
- Stars
- 10
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
`loop_tasks()` can be quite slow for large task makefiles. A current hypothesis is that a major cause of this slowness is that `scmake` gets (called separately for every target)[https://github.com/USGS-R/scipiper/blob/e5e90b11200aaf4448683889c7bd0ea487b6e9dd/R/loop_tasks.R#L138], and that this is slow because `scmake` calls `remake` rereads the task .yml file and rebuilds the dependency tree every time.
For this task, find a way to only read the task .yml file once within that loop over `incomplete_targets`. This might involve commands that dig into the remake code a bit, perhaps things like
```
remake_object <- remake:::remake(remake_file=remake_file, verbose=FALSE, load_sources=TRUE)
```
which could be called just once and then called to some other remake function that does the making based on that remake_object. Look at the source code for `remake::make` to figure out what is needed and which internal functions might be called.
Contributor guide
Assessment
This issue has not been assessed yet.