googleapis / googleapis/google-cloud-python

Add DBAPI executemany INSERT batching support

Abierto
#15,700 5 comentarios 0 reacciones 1 asignado Reclamado por @daniel-sanche Ver en GitHub
api: bigquery priority: p3 type: feature request
Lenguaje dominante
Python
Estrellas
5.4k
Forks
1.8k
Merge medio
3 d 4 h
PR fusionados (30 d)
122

Descripción

**Is your feature request related to a problem? Please describe.**
It appears that the BigQuery DBAPI does not support multi-row INSERT batching for more performant python-based DML transactions. Current executemany INSERT statements are executed one at a time, leading to massive slowdowns in batch INSERT DML operations.

Example multi-row insert from BiqQuery documentation:
https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#insert_examples
```
INSERT dataset.Inventory (product, quantity)
VALUES('top load washer', 10),
('front load washer', 20),
('dryer', 30),
('refrigerator', 10),
('microwave', 20),
('dishwasher', 30),
('oven', 5)
```
 
**Describe the solution you'd like**
Add multi-row INSERT batching support.
MySQL DBAPI example: https://github.com/PyMySQL/PyMySQL/blob/main/pymysql/cursors.py#L194

**Describe alternatives you've considered**
I will probably crudely make a patch to my sqlalchemy-bigquery DBAPI cursor to enable this support for my project that needs this performance boost for my ORM based application.

**Additional context**
sqlalchemy-bigquery related ticket: https://github.com/googleapis/google-cloud-python/issues/15795
sqlalchemy related discussion: https://github.com/sqlalchemy/sqlalchemy/discussions/12038

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.