GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst
Use GCP bucket for temp_location when running locally (dataflow)
- Dominant language
- Jupyter Notebook
- Stars
- 8.6k
- Forks
- 6.1k
- Avg merge
- 4h 44m
- Merged PRs (30d)
- 2
Description
In `training-data-analyst/courses/machine_learning/deepdive2/end_to_end_ml/labs/preproc.ipynb`, there are two options:
- Running and storing locally (when `in_test_mode=True`; `OUTPUT_DIR = './preproc'` -- local directory)
- Running and storing on GCP (when `in_test_mode=False`; `OUTPUT_DIR='gs://{0}/babyweight/preproc/'.format(BUCKET)` -- GCS bucket)
However, when running locally, the `temp_location` needs to be a GCS bucket instead of a local directory, otherwise it will result in an error:
` "error": {
"code": 400,
"message": "Invalid extract destination URI './preproc/tmp/18bcf98db67c4b1c9a6313bc465f137f/bigquery-table-dump-*.json'. Must be a valid Google Cloud Storage path and filename/pattern.",
"errors": [
{
"message": "Invalid extract destination URI './preproc/tmp/18bcf98db67c4b1c9a6313bc465f137f/bigquery-table-dump-*.json'. Must be a valid Google Cloud Storage path and filename/pattern.",
"domain": "global",
"reason": "invalid"
}
],
"status": "INVALID_ARGUMENT"
}
`
Contributor guide
Assessment
This issue has not been assessed yet.