Datalink getdataset too eager
- Dominant language
- Python
- Stars
- 88
- Forks
- 64
- Avg merge
- 38m
- Merged PRs (30d)
- 1
Description
Consider this script:
```
import pyvo
ACCESS_URL = "http://dc.g-vo.org/lswscans/res/positions/siap/siap.xml?"
svc = pyvo.sia.SIAService(ACCESS_URL)
images = svc.search((11,35), size=(0.1, 0.1))
images[0].cachedataset()
```
Warning: This will download at least 100 Megabytes with current pyVO.
It shouldn't, though.
This is one of the SIA1 services that return cutouts as access_urls. However, there is a datalink service on the result, too. That service only uses the full data ids, not any cutout instructions. That is, #this can be a FITS of 1 GB or so, whereas the accref can point to just a few epsilons.
Now, you can argue that if I can't make #this exactly the dataset that is described in the SIA row then I shouldn't make the datalink in the first place; but I claim this arrangement is useful in practice (e.g., people can adjust their cutouts, or pull the full image if that's what they want) and at least not *totally* wrong.
On the other hand, pyVO's current behaviour, that is, to prefer datalink's #this over the access reference given in the SIAP response, is weird (to say the least). I'm not even sure why we're doing it. It seems the original code was written by @funbaker, who I think has dropped out of pyVO development, so I don't think we can ask him.
Anyway, I'd suggest to prefer SIAP's access reference and only fall back to datalink if that's not going anywhere. Technically, I'd simply turn around the logic in adhoc.DatalinkRecordMixin.getdataset: try super() first and only if that exceptions out try datalink's #this.
Does anyone feel that would be grossly wrong?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with adhoc.DatalinkRecordMixin.getdataset and inspect the superclass path used for the SIAP access reference. Reproduce the provided SIAService example, then verify that the SIAP access reference is preferred and datalink's #this is used only when the former fails, without eagerly downloading the larger dataset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100