dbt-labs / dbt-labs/dbt-adapters
[Feature] Add enable_fine_grained_mutations support BigQuery
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this your first time submitting a feature request?
- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
This feature can improve the speed of UPDATE, DELETE, and MERGE queries which could be helpful with incremental dbt model with the merge strategy as well as potential implementations of GDPR processes.
[As they say themselves](https://cloud.google.com/blog/products/data-analytics/bigquery-features-for-transactional-data-management):
> When enabled, instead of always rewriting large storage blocks, BigQuery fine-grained DML can pinpoint and modify data with much finer granularity. It leverages optimized metadata indexes to rewrite only the necessary mutated data, drastically reducing the processing, I/O, and consequently, the slot time consumed for sparse DML. The result? Faster, more cost-effective DM
### Describe alternatives you've considered
I considered including it in exactly the same way as the `enable_change_history` in this PR: #1225
These features are both described in the [blog](https://cloud.google.com/blog/products/data-analytics/bigquery-features-for-transactional-data-management).
### Who will this benefit?
Incremental merge queries could see increased performance.
GDPR implementations that anonymize or delete data in tables created by dbt.
### Are you interested in contributing this feature?
Yes. I have adapted the #1225 for this option
### Anything else?
https://cloud.google.com/blog/products/data-analytics/bigquery-features-for-transactional-data-management
https://cloud.google.com/bigquery/docs/data-manipulation-language#fine-grained_dml
Contributor guide
Assessment
This issue has not been assessed yet.