astropy / astropy/astroquery

504 Server Error in X-Match query

Open
#1,900 15 comments 0 reactions 0 assignees View on GitHub
xmatch
Dominant language
Python
Stars
791
Forks
451
Avg merge
1d 3h
Merged PRs (30d)
4

Description

I have been using astroquery's wrapper around X-Match to run crossmatches against big surveys, and strangely it fails for certain surveys which is new behaviour. I see the following error:

` d = XMatch.query(cat1=open(masterlist),cat2='vizier:%s' % code, max_distance=angular_sep * u.arcsec,colRA1='ra',colDec1='dec')
File "/Users/moss/anaconda/lib/python3.6/site-packages/astroquery/xmatch/core.py", line 61, in query
get_query_payload=get_query_payload)
File "/Users/moss/anaconda/lib/python3.6/site-packages/astroquery/xmatch/core.py", line 95, in query_async
response.raise_for_status()
File "/Users/moss/anaconda/lib/python3.6/site-packages/requests/models.py", line 939, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: http://cdsxmatch.u-strasbg.fr/xmatch/api/v1/sync`

I checked the self.TIMEOUT parameter passed to the requests call, which was 300, but I tried setting it to much higher and that made no difference to the error, it still timed out and refused to continue. This was specifically for querying against the ALLWISE catalogue, whereas for other catalogues (e.g. SUMSS, XMMDR8, etc) it finishes within about a minute. It's possible this is not an astroquery issue but traces back to X-Match, but I found when I did a crossmatch on the X-Match website (against 2MASS, an equally big catalogue) it took a long time but actually did seem to work okay. Wondering if there is any kind of fix possible on the astroquery side? Thanks in advance!

Contributor guide

Open the contributing guide

Research direction

Start in astroquery/xmatch/core.py, especially query_async, and reproduce the ALLWISE request with the shown parameters while checking the service response and timeout behavior. Done means establishing whether the 504 is caused by astroquery or the X-Match service and documenting or implementing a justified client-side fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.