astropy / astropy/pyvo

ENH: Add VOParquet support for TAP queries

Open
#753 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
88
Forks
64
Avg merge
38m
Merged PRs (30d)
1

Description

I am wondering if it would be worth adding support for VOParquet (https://www.ivoa.net/documents/Notes/VOParquet/20250116/NOTE-voparquet-1.0-20250116.html)?

This is supported as far as I can see in astropy, see:
```
job = AsyncTAPJob.create(
service.baseurl, query,
RESPONSEFORMAT="application/vnd.apache.parquet",
session=session
)
job = job.run().wait()
response = session.get(job.result_uri, stream=True)
table = Table.read(io.BytesIO(response.content),
format="parquet.votable")
```

At Rubin we do currently support `VOParquet` output and are generally eager to start advertising that option more because of the noticeable performance improvements it gives us, so it would be nice if this was possible through PyVO.

If folks think this is worth adding I'm happy to work on this if no one else already was planning to do so.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with AsyncTAPJob.create and the TAP result-handling path shown in the issue. Compare the application/vnd.apache.parquet response with astropy's Table.read(..., format="parquet.votable") example. Done means PyVO can request VOParquet for TAP queries and correctly expose the returned table.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.