DOI-USGS / DOI-USGS/scipiper

require_local

Open
#101 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
10
Forks
12
PR merge metrics
No merged PRs in 30d

Description

If we like it after some experimentation, let's import `require_local` from the lake temperature pipelines into scipiper. Example: https://github.com/USGS-CIDA/lake-temperature-neural-networks/blob/1a39fea4c7ef8d3f6e9374fcd720e2c9d82351d1/lib/src/require_local.R#L1

The idea of this function is to allow us to have a chain of .ind files, each depending on the next, without requiring that the corresponding data_files all exist on the shared cache OR that all collaborators build/have the file locally. Only collaborators who need to use the contents should need to build it. This makes sense for data files that are intermediates on the way to data files that multiple people actually want to use (without caring to ever look at the intermediate data files).

It works by accepting an indicator file and throwing an error if the corresponding data file is not locally present. The error recommends that you rebuild the ind_file on the assumption that you have pulled the ind_file from github, it was doing its job of reassuring remake that the file had been built sufficiently for our purposes (in this case, on someone else's machine), and now you're asking more of it (to actually guarantee the data file's presence on _your_ computer rather than just somewhere on _somebody's_ computer.

Contributor guide

Open the contributing guide

Research direction

Start by reading the referenced require_local.R implementation and then inspect scipiper's handling of .ind files and corresponding data files. Verify the behavior with a chained set of indicator files: collaborators should be able to use pulled indicators without local intermediates, while a missing local data file should produce an error recommending that its indicator be rebuilt.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.