astropy / astropy/astroquery

Not overwriting MAST download of incomplete file

Open
#3,231 9 comments 0 reactions 0 assignees View on GitHub
bug mast
Dominant language
Python
Stars
791
Forks
451
Avg merge
1d 3h
Merged PRs (30d)
4

Description

I believe astroquery.mast should be able to download and write over an incomplete file. However, it still is corrupted/incomplete even when the code recognizes the file as incomplete. Here are the steps to re-produce.

from astroquery.mast import Observations
from astropy.io import fits
fileN = 'jw01185103001_02102_00001-seg001_nrcalong_rate.fits'
with open(fileN,'wb') as f:
f.write(b'junk')
Observations.download_file('mast:jwst/product/'+fileN)
tmpHDU = fits.open(fileN)

I get `OSError: Empty or corrupt FITS file` whereas I was expecting astroquery to download a new file because it detects that the cached one is incomplete.

I'm using astroquery 0.4.10.dev9927 on Python 3.12.2

Contributor guide

Open the contributing guide

Research direction

Start at the Observations.download_file entry point and reproduce the example with a four-byte file named jw01185103001_02102_00001-seg001_nrcalong_rate.fits. Trace the incomplete-file detection and download path; done means the cached file is replaced and fits.open(fileN) succeeds without an empty or corrupt FITS error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.