STSCI: Truncation warning appears when results are not truncated
- Dominant language
- Python
- Stars
- 88
- Forks
- 64
- Avg merge
- 38m
- Merged PRs (30d)
- 1
Description
When uing pyvo to search for JWST data, I'm running into a situation where I'm getting a warning that the results are truncated when it turns out that they aren't.
Here is the code I am using:
```
import pyvo as vo
instrument = 'miri'
proposal = 9233
STSCI_VO_URL = "https://mast.stsci.edu/vo-tap/api/v0.1/caom"
tap_service = vo.dal.TAPService(STSCI_VO_URL)
tap_results = tap_service.search(f"select observationID from dbo.CaomObservation where collection='JWST' and insName like '{instrument.lower()}%' and prpID='{int(proposal)}'")
```
and the result is:
```
site-packages/pyvo/dal/query.py:404: DALOverflowWarning: Results truncated due to server limits. Consider setting a maxrec value.
warn("Results truncated due to server limits. Consider "
len(tap_results)
102
```
If I repeat the query using a value of maxrec equal to 0 or to any value >102, I see the truncation warning and the results show the same list of 102 filenames. If I set maxrec to a value between 0 and 102, then the truncation warning does not show up, and the length of the resulting list is limited to maxrec entries, as expected. So it seems like the truncation warning should not be showing up in cases where maxrec > 102 (or 0)?
Contributor guide
No contributing guide indexed for this repository
Research direction
Inspect site-packages/pyvo/dal/query.py around line 404, where DALOverflowWarning is emitted, and reproduce the TAPService query with maxrec set to 0, 102, and a value above 102. Confirm the warning is only reported when results are actually truncated, while preserving the expected result lengths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- api, data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100