GoogleCloudPlatform / GoogleCloudPlatform/bigquery-utils

Feature Request: Authorized View Generator / Refresher script

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
1.3k
Forks
337
PR merge metrics
No merged PRs in 30d

Description

Often customers have a source table and an authorized view which drops a few blacklisted fields.
Sometimes the schema of the underlying table changes and the view definition drops fields it shouldn't.

Let's create a script that will pull the schema from an existing table and drop a list of blacklisted fields (or sub-fields) and generate the SQL for a view.

The script should take the following inputs:
- source table (fully qualified project.dataset.table)
- destination view ( fully qualified project.dataset.view)
- list of fields to drop

Considerations:
- Handle dropping fields from within structs in a nested schema.
- Implementing this in Golang would be nice so it can be a compiled CLI utility and for ease of usage from terraform modules.

Nice to Have:
- Include automation to Create the view and reinstate it's authorization on the source dataset.

Ugly Python Prototype: [gist](https://gist.github.com/jaketf/dc7c0e4506fec1f0048e37e643775b02)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.