googleapis / googleapis/google-cloud-python

Add DBAPI executemany INSERT batching support

Đang mở
#15,700 5 bình luận 0 reaction 1 người được giao Được @daniel-sanche nhận Xem trên GitHub
api: bigquery priority: p3 type: feature request
Ngôn ngữ chính
Python
Star
5.4k
Fork
1.8k
Merge trung bình
3 ngày 4 giờ
Pull request đã merge (30 ngày)
122

Mô tả

**Is your feature request related to a problem? Please describe.**
It appears that the BigQuery DBAPI does not support multi-row INSERT batching for more performant python-based DML transactions. Current executemany INSERT statements are executed one at a time, leading to massive slowdowns in batch INSERT DML operations.

Example multi-row insert from BiqQuery documentation:
https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#insert_examples
```
INSERT dataset.Inventory (product, quantity)
VALUES('top load washer', 10),
('front load washer', 20),
('dryer', 30),
('refrigerator', 10),
('microwave', 20),
('dishwasher', 30),
('oven', 5)
```
 
**Describe the solution you'd like**
Add multi-row INSERT batching support.
MySQL DBAPI example: https://github.com/PyMySQL/PyMySQL/blob/main/pymysql/cursors.py#L194

**Describe alternatives you've considered**
I will probably crudely make a patch to my sqlalchemy-bigquery DBAPI cursor to enable this support for my project that needs this performance boost for my ORM based application.

**Additional context**
sqlalchemy-bigquery related ticket: https://github.com/googleapis/google-cloud-python/issues/15795
sqlalchemy related discussion: https://github.com/sqlalchemy/sqlalchemy/discussions/12038

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.