GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst

Use GCP bucket for temp_location when running locally (dataflow)

Open
#1,348 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
8.6k
Forks
6.1k
Avg merge
4h 44m
Merged PRs (30d)
2

Description

In `training-data-analyst/courses/machine_learning/deepdive2/end_to_end_ml/labs/preproc.ipynb`, there are two options:

- Running and storing locally (when `in_test_mode=True`; `OUTPUT_DIR = './preproc'` -- local directory)
- Running and storing on GCP (when `in_test_mode=False`; `OUTPUT_DIR='gs://{0}/babyweight/preproc/'.format(BUCKET)` -- GCS bucket)

However, when running locally, the `temp_location` needs to be a GCS bucket instead of a local directory, otherwise it will result in an error:

` "error": {
"code": 400,
"message": "Invalid extract destination URI './preproc/tmp/18bcf98db67c4b1c9a6313bc465f137f/bigquery-table-dump-*.json'. Must be a valid Google Cloud Storage path and filename/pattern.",
"errors": [
{
"message": "Invalid extract destination URI './preproc/tmp/18bcf98db67c4b1c9a6313bc465f137f/bigquery-table-dump-*.json'. Must be a valid Google Cloud Storage path and filename/pattern.",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.