CADC Grabbing cutouts from a get_image_list URL
- Dominant language
- Python
- Stars
- 791
- Forks
- 451
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 4
Description
Hi, I've written a small bit of code to query the CADC archive for HST data cutout. I use get_image_list to retrieve the image filenames, but I've noticed that the code I came up with to grab the fits data cutout isn't always reliable for all queries. Sometimes I get back an empty list. I've traced this to a HTTP 409 error that I seem to be getting. Any clues on what might be going wrong here?
A minimal example:
```
import requests
def get_images_from_list(image_list, show_progress=False):
"""Returns lists of HDUs from given image list URLS"""
from astroquery.utils import commons
fits_list = []
filenames = [commons.FileContainer(url, encoding='binary',show_progress=show_progress) for url in image_list]
for fn in filenames:
try:
fits_list.append(fn.get_fits())
except requests.exceptions.HTTPError as err:
# Catch HTTPError if user is unauthorized to access file
print("{} - Problem retrieving the file: {}".format(str(err), str(err.url)))
pass
return fits_list
#This URL fails
image_list_fail = ['https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/sync?ID=mast%3AHST%2Fproduct%2Fj8xi6ajdq_flc.fits&RUNID=p8dbfo12deijxaag&POS=CIRCLE+150.298095703125+2.7121429443359375+0.0015277777777777779']
#This URL doesn't
image_list_pass = ['https://www.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/caom2ops/sync?ID=mast%3AHST%2Fproduct%2Fjcor23010_drz.fits&RUNID=chsl0cmf1fwzq90d&POS=CIRCLE+150.33006909892683+2.07031305099778+0.0015277777777777779']
#Grab fits images
fits_list = get_images_from_list(image_list_pass, show_progress=False)
print("Success!")
fits_list = get_images_from_list(image_list_fail, show_progress=False)
```
Contributor guide
Research direction
Start with the minimal Python example and compare the failing and passing CADC sync URLs, focusing on the HTTP 409 response and the use of get_image_list and commons.FileContainer.get_fits(). Reproduce both requests and determine what change would make retrieval reliable for the failing query; done means the cutout is returned instead of an empty list.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100