googleapis / googleapis/google-cloud-python
feat!: convert arrow data types to Python types in `rows()`
- Langage dominant
- Python
- Étoiles
- 5.4k
- Forks
- 1.8k
- Merge moyen
- 3 j 4 h
- PR mergées (30 j)
- 122
Description
I notice we just loop through all the rows in a record batch here:
https://github.com/googleapis/python-bigquery-storage/blob/f7fc4ec45e62739139bf865497ce2e5fbd948d31/google/cloud/bigquery_storage_v1/reader.py#L695
This might result in some odd types, such as [Int64Scalar](https://arrow.apache.org/docs/python/generated/pyarrow.Int64Scalar.html?highlight=scalar#pyarrow.Int64Scalar) and [TimestampScalar](https://arrow.apache.org/docs/python/generated/pyarrow.TimestampScalar.html?highlight=scalar#pyarrow.TimestampScalar) sneaking in.
I think we probably want to call [to_pydict](https://arrow.apache.org/docs/python/generated/pyarrow.RecordBatch.html?highlight=recordbatch#pyarrow.RecordBatch.to_pydict) before looping over rows/columns in the BigQuery Storage API client.
I'm filing this issue in `python-bigquery` at first, because I think it's worth investigating if this is actually happening first.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.