googleapis / googleapis/google-cloud-python

Schema conflict when storing dataframes with datetime objects using load_table_from_dataframe()

Ouverte
#15,743 24 commentaires 0 réactions 2 personnes assignées Réclamée par @chalmerlowe Voir sur GitHub
api: bigquery priority: p3 type: bug
Langage dominant
Python
Étoiles
5.4k
Forks
1.8k
Merge moyen
3 j 4 h
PR mergées (30 j)
122

Description

The output

Traceback (most recent call last):
File "/tmp/mwe.py", line 14, in
print(job.result())
File "/home/tatome/anaconda3/envs/default_env/lib/python3.6/site-packages/google/cloud/bigquery/job.py", line 688, in result
return super(_AsyncJob, self).result(timeout=timeout)
File "/home/tatome/anaconda3/envs/default_env/lib/python3.6/site-packages/google/api_core/future/polling.py", line 120, in result
raise self._exception
google.api_core.exceptions.BadRequest: 400 Provided Schema does not match Table [project+dataset].test. Field test_column has changed type from DATETIME to TIMESTAMP.

is produced by the following code:

import pandas as pd
from google.cloud.bigquery import Client
client = Client(project='[project]')
table_name = 'test'
job = client.query(f'CREATE TABLE [dataset].{table_name} ( idx INT64, test_column DATETIME )')
job.result()
df = pd.DataFrame({'idx' : [0,1], 'test_column' : ['2018-12-24', '2019-01-01']})
df['test_column'] = pd.to_datetime(df['test_column']).values.astype('datetime64[ms]')
dataset_ref = client.dataset('[dataset]')
table_ref = dataset_ref.table(table_name)
df = df.set_index('idx')
job = client.load_table_from_dataframe(df, table_ref, location='EU')
print(job.result())

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.