astropy / astropy/astroquery

More robust file download mechanism in cadc

Open
#1,719 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
791
Forks
451
Avg merge
1d 3h
Merged PRs (30d)
4

Description

If present, the mechanism will use the `content-MD5` or `contentLength` headers to check if file needs to be downloaded and to check the integrity of the downloaded data. It will also implement some basic re-trials on transient HTTP errors.

If it proves generic enough it could be considered for the generic `commons` module.

Contributor guide

Open the contributing guide

Research direction

Start by locating the cadc file-download mechanism and reading how it currently fetches and validates data. The work should use content-MD5 or contentLength when available, retry transient HTTP errors, and leave the mechanism generic enough for possible reuse in the commons module.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.