googleapis / googleapis/google-cloud-python

Support INTERVAL data type in DB-API, arrow, and pandas connectors

オープン
#15,739 コメント 5 件 リアクション 1 件 担当者 1 名 @tswast が担当を希望しています GitHub で見る
api: bigquery external status: blocked type: feature request
主要言語
Python
スター
5.4k
フォーク
1.8k
平均マージ
3日 4時間
マージ済み PR(30日)
122

説明

Follow-up to https://github.com/googleapis/google-cloud-python/issues/15740 since pandas and Arrow do not yet have a structured type that aligns with `INTERVAL`. The existing [Timedelta](https://pandas.pydata.org/docs/reference/api/pandas.Timedelta.html) support would work for INTERVALS with only a time component, but it is not calendar aware, so supporting year, month, and day intervals would require some mapping to timedelta, which is not ideal.

Why is a new data type needed?

* YEAR: Leap years are a thing. Not every year is 365 days long.
* MONTH: Not every month is the same length.
* DAY: Daylight savings is a thing. Not every day is 24 hours long.

Note: DB-API support is included here because it uses the BigQuery Storage API, where we use the Arrow wire format.

TODO:

- [ ] Auto-detect data type in DB-API query parameters
- Might be possible to do this before reading INTERVAL columns is supported.
- [ ] Row data is converted to relevant type in DB-API
- [ ] Row data is converted to relevant type in `to_dataframe`
- Might need to be `object`, since `timedelta64` doesn't have years/months.
- [ ] Check if `to_arrow` type is expected datatype
- [ ] Convert data type in `insert_rows_from_dataframe`
- [ ] Convert data type in `load_rows_from_dataframe` (CSV)
- [ ] Convert data type in `load_rows_from_dataframe` (Parquet)

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。