googleapis / googleapis/google-cloud-go
bigquery: support extended storage API options for query acceleration
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 1.6k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 109
Description
## Is your feature request related to a problem? Please describe.
To get best Arrow read performance from BigQuery Storage Read, we need to be able to enable Arrow compression. We noticed this because the Python client gets nearly twice as much bandwidth as the Go client, and after some benchmarking and trying different things, it turned out that enabling compression was the key difference.
## Describe the solution you'd like
Either always enable compression, or provide an option somewhere to enable compression and choose the type (LZ4 or ZSTD).
## Describe alternatives you've considered
We could fork the client...
## Additional context
In storage_client.go we see `readSession.start` does not support any way of setting `arrow_serialization_options` in `CreateReadSessionRequest`.
https://github.com/googleapis/google-cloud-go/blob/a64705b6d62f4b4e42b870df55d8937b7836b8b7/bigquery/storage_client.go#L138-L165
Contributor guide
Assessment
This issue has not been assessed yet.