GoogleCloudPlatform / GoogleCloudPlatform/tensorflow-lifetime-value

Incorrect PREFIX_JOBS_EXPORT location

Open
#1 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
174
Forks
72
PR merge metrics
No merged PRs in 30d

Description

When I tried to run AIrflow job, copy_model_to_final failed with error as the GCS bucket has no corresponding folder.

https://github.com/GoogleCloudPlatform/tensorflow-lifetime-value/blob/6534bcf8aa983c5bb754c3b2973cc07ee7687341/run/airflow/dags/01_build_train_deploy.py

```
def do_copy_model_to_final(**kwargs):
gcs = GoogleCloudStorageHook()

# Returns all the objects within the bucket. All sub-buckets are considered
# as prefix of the leaves. List does not differentiate files from subbuckets
all_jobs_files = gcs.list(
bucket=COMPOSER_BUCKET_NAME,
prefix='{}/export/estimate'.format(PREFIX_JOBS_EXPORT)
```

Since the previous job export the model under gs://_composer_final/jobs/clv-composer/clv-201903101422/export we should omit "clv-datetime" folder name to detect the folder correctly.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.