googleapis / googleapis/google-cloud-python

Support extended range of pd.Timestamp with pandas 2.0

Abierto
#15,661 3 comentarios 0 reacciones 1 asignado Reclamado por @ivansmf Ver en GitHub
api: bigquery bigframes priority: p3 type: feature request
Lenguaje dominante
Python
Estrellas
5.4k
Forks
1.8k
Merge medio
3 d 4 h
PR fusionados (30 d)
122

Descripción

**Is your feature request related to a problem? Please describe.**

When querying TIMESTAMP column, with pandas > 2.0 installed, the dtype is 'datetime64[us, UTC]`.
This is using the same precision as bigquery, which is what we want.

However if I query a TIMESTAMP column that contains data that is out of bounds for a 'datetime64[ns, UTC]` (but fit in a 'datetime64[us, UTC]`), the column dtype becomes `object`.

This is inefficient and also we lose meta data here.

**Describe the solution you'd like**

If pandas > 2.0, always return `datetime64[us, UTC]` for timestamp column

**Additional context**

The same thing happens for DATE and DATETIME and should be addressed as well.

This test captures the current behavior: https://github.com/googleapis/python-bigquery-pandas/blame/72f772ec7b36ba081ed18ef4409f8262a412a1b9/tests/system/test_read_gbq.py#L538

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.