CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator
Recycle old download files when available
- Dominant language
- Python
- Stars
- 12
- Forks
- 3
- Avg merge
- 3d 3m
- Merged PRs (30d)
- 9
Description
The download log stores the parameters and filename of each request, so it can be used as a caching system.
Roughly speaking, if there is a download in the database with the same corpus + parameters as a new request, you can just log the same filename and skip generating a new file.
It would be good to also build in some expiration date, e.g. only do this if the original download was started within the last n days. While the old file can remain available to the user that requested it originally, new users should not receive possible outdated data. (Due to updates or the like.)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.