GoogleCloudPlatform / GoogleCloudPlatform/professional-services

BQ dataset used in dataflow example doesn't exist

Open
#527 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.1k
Forks
1.5k
Avg merge
2d 8h
Merged PRs (30d)
44

Description

The `python-dataflow-example.example_data` BigQuery dataset used in the cookbook batch-examples of dataflow-examples code is returning a 404 and all operations with code that references it are failing.

Did this dataset change in some way recently?

Link to two files where I know dataset is referenced:
https://github.com/GoogleCloudPlatform/professional-services/blob/master/examples/dataflow-python-examples/batch-examples/cookbook-examples/pipelines/data_enrichment.py
https://github.com/GoogleCloudPlatform/professional-services/blob/master/examples/dataflow-python-examples/batch-examples/cookbook-examples/pipelines/data_lake_to_mart.py

Contributor guide

Open the contributing guide

Research direction

Start with the referenced pipelines/data_enrichment.py and pipelines/data_lake_to_mart.py files, then inspect how they reference the python-dataflow-example.example_data BigQuery dataset. Verify the 404 and determine whether the examples need a valid replacement or updated reference; done means the affected operations run successfully against an available dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.