googleapis / googleapis/google-cloud-python
feat!: convert arrow data types to Python types in `rows()`
- 主要语言
- Python
- 星标
- 5.4k
- 派生
- 1.8k
- 平均合并
- 3 天 4 小时
- 30 天内合并 PR
- 122
描述
I notice we just loop through all the rows in a record batch here:
https://github.com/googleapis/python-bigquery-storage/blob/f7fc4ec45e62739139bf865497ce2e5fbd948d31/google/cloud/bigquery_storage_v1/reader.py#L695
This might result in some odd types, such as [Int64Scalar](https://arrow.apache.org/docs/python/generated/pyarrow.Int64Scalar.html?highlight=scalar#pyarrow.Int64Scalar) and [TimestampScalar](https://arrow.apache.org/docs/python/generated/pyarrow.TimestampScalar.html?highlight=scalar#pyarrow.TimestampScalar) sneaking in.
I think we probably want to call [to_pydict](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html?highlight=recordbatch#pyarrow.RecordBatch.to_pydict) before looping over rows/columns in the BigQuery Storage API client.
I'm filing this issue in `python-bigquery` at first, because I think it's worth investigating if this is actually happening first.
贡献指南
评估
这个 Issue 还没有评估数据。