astropy / astropy/pyvo

STSCI: Truncation warning appears when results are not truncated

Open
#735 2 comments 1 reaction 0 assignees View on GitHub
upstream
Dominant language
Python
Stars
88
Forks
64
Avg merge
38m
Merged PRs (30d)
1

Description

When uing pyvo to search for JWST data, I'm running into a situation where I'm getting a warning that the results are truncated when it turns out that they aren't.

Here is the code I am using:
```
import pyvo as vo
instrument = 'miri'
proposal = 9233
STSCI_VO_URL = "https://mast.stsci.edu/vo-tap/api/v0.1/caom"
tap_service = vo.dal.TAPService(STSCI_VO_URL)
tap_results = tap_service.search(f"select observationID from dbo.CaomObservation where collection='JWST' and insName like '{instrument.lower()}%' and prpID='{int(proposal)}'")
```
and the result is:
```
site-packages/pyvo/dal/query.py:404: DALOverflowWarning: Results truncated due to server limits. Consider setting a maxrec value.
warn("Results truncated due to server limits. Consider "

len(tap_results)
102
```

If I repeat the query using a value of maxrec equal to 0 or to any value >102, I see the truncation warning and the results show the same list of 102 filenames. If I set maxrec to a value between 0 and 102, then the truncation warning does not show up, and the length of the resulting list is limited to maxrec entries, as expected. So it seems like the truncation warning should not be showing up in cases where maxrec > 102 (or 0)?

Contributor guide

No contributing guide indexed for this repository

Research direction

Inspect site-packages/pyvo/dal/query.py around line 404, where DALOverflowWarning is emitted, and reproduce the TAPService query with maxrec set to 0, 102, and a value above 102. Confirm the warning is only reported when results are actually truncated, while preserving the expected result lengths.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sql
Domain
api, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.