GoogleCloudPlatform / GoogleCloudPlatform/tensorflow-lifetime-value
Incorrect PREFIX_JOBS_EXPORT location
- Dominant language
- Jupyter Notebook
- Stars
- 174
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
When I tried to run AIrflow job, copy_model_to_final failed with error as the GCS bucket has no corresponding folder.
https://github.com/GoogleCloudPlatform/tensorflow-lifetime-value/blob/6534bcf8aa983c5bb754c3b2973cc07ee7687341/run/airflow/dags/01_build_train_deploy.py
```
def do_copy_model_to_final(**kwargs):
gcs = GoogleCloudStorageHook()
# Returns all the objects within the bucket. All sub-buckets are considered
# as prefix of the leaves. List does not differentiate files from subbuckets
all_jobs_files = gcs.list(
bucket=COMPOSER_BUCKET_NAME,
prefix='{}/export/estimate'.format(PREFIX_JOBS_EXPORT)
```
Since the previous job export the model under gs://_composer_final/jobs/clv-composer/clv-201903101422/export we should omit "clv-datetime" folder name to detect the folder correctly.
Contributor guide
Assessment
This issue has not been assessed yet.