dbt-labs / dbt-labs/dbt-adapters

[CT-1710] [CT-1703] [Feature] `on_schema_change` should handle non-top-level schema changes

Open
#599 12 comments 10 reactions 0 assignees View on GitHub
feature:nested-columns help-wanted pkg:dbt-bigquery type:enhancement
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this your first time submitting a feature request?

- [X] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [X] I have searched the existing issues, and I could not find an existing issue for this feature
- [X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

### Describe the feature

Currently, `on_schema_change` tracks schema changes involving top-level columns. This excludes nested changes, like the addition of a column in a `STRUCT` (BigQuery). It would be great to extend the functionality of `on_schema_change` to track changes in the schema for top-level columns as well as nested ones, too.

### Describe alternatives you've considered

We've had to update schemas manually to accommodate nested changes. Or, we write the data to new tables and then migrate those new tables to a production context.

### Who will this benefit?

This type of feature will benefit those with meaty incremental models who do not want to or cannot afford to do a full refresh. This benefits those with models that change somewhat frequently - more holistic, automated handling of schema changes would be fantastic!

### Are you interested in contributing this feature?

Sure! If you all see this as a feasible feature extension, I am happy to dig into the code and contribute.

### Anything else?

At the very least, I think the documentation around `on_schema_change` should be updated to make it very clear that it deals with top-level schema changes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.