More robust file download mechanism in cadc
Open
- Dominant language
- Python
- Stars
- 791
- Forks
- 451
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 4
Description
If present, the mechanism will use the `content-MD5` or `contentLength` headers to check if file needs to be downloaded and to check the integrity of the downloaded data. It will also implement some basic re-trials on transient HTTP errors.
If it proves generic enough it could be considered for the generic `commons` module.
Contributor guide
Research direction
Start by locating the cadc file-download mechanism and reading how it currently fetches and validates data. The work should use content-MD5 or contentLength when available, retry transient HTTP errors, and leave the mechanism generic enough for possible reuse in the commons module.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100