improve efficiency of loop_tasks is_current check
- Dominant language
- R
- Stars
- 10
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
I was just looking at https://github.com/USGS-R/scipiper/blob/master/R/loop_tasks.R#L70-L76 again and am not sure it's as efficient as I initially thought. In particular, we call `get_remake_status(targets, task_makefile)` on _all_ possible targets and only later exclude those that already have an existing file. But the check for whether existing files are current is the slow part, so we surely shouldn't be calling `get_remake_status(targets, task_makefile)` on those existing files at all, right?
And maybe we should make it a user option to either loop over all tasks or to loop over complete unbuilt tasks (for which file doesn't yet exist)? Sometimes the hashing is expensive while the retries of unbuilt files are the focus; other times the hashing is cheap and we'd like to include retries for built but out-of-date files (e.g., when we forget or don't know to or don't want to delete the files before building).
Contributor guide
Assessment
This issue has not been assessed yet.