bcgov / bcgov/nr-rfc-processing
Modify Daily pipeline so fills in missing data for a given time window
Open
bug
enhancement
- Dominant language
- Python
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
currently the script is setup to run on a daily basis where it runs using the current date.
Issues with this approach is the data is not always available for the current date. Sometimes there can be a 3-4 day lag before data becomes available.
Changes:
* instead of just running a specific date, the script will check what data exists in the object store bucket vs the data that is currently available. It will then run the pipeline for consecutive days, untill al the data that is currently available has been processed.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.