GoogleCloudPlatform / GoogleCloudPlatform/realtime-embeddings-matching

Error in running the command bash run.sh

Open
#3 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
129
Forks
29
PR merge metrics
No merged PRs in 30d

Description

Hi,
I am getting the following error while running the command bash run.sh

Cleaning working and output directories...
CommandException: "rm" command does not support "file://" URLs. Did you mean to use a gs:// URL?
CommandException: "rm" command does not support "file://" URLs. Did you mean to use a gs:// URL?
Running the Dataflow job...
********************************************************************************
Python 2 is deprecated. Upgrade to Python 3 as soon as possible.
See https://cloud.google.com/python/docs/python2-sunset
To suppress this warning, create an empty ~/.cloudshell/no-python-warning file.
The command will automatically proceed in seconds or on any key.
********************************************************************************
Traceback (most recent call last):
File "run.py", line 69, in
main()
File "run.py", line 64, in main
pipeline.run(pipeline_options, known_args)
File "/home/shahmansi78/realtime-embeddings-matching/text-semantic-search/embeddings_extraction/etl/pipeline.py", line 103, in run
pipeline = beam.Pipeline(options=pipeline_options)
File "/home/shahmansi78/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 149, in __init__
'Pipeline has validations errors: \n' + '\n'.join(errors))
ValueError: Pipeline has validations errors:
Invalid GCS path (wordembedding/wikipedia/dataflow/temp), given for the option: temp_location.
Invalid GCS path (wordembedding/wikipedia/dataflow/staging), given for the option: staging_location.
Dataflow job submitted successfully!

Contributor guide

Open the contributing guide

Research direction

Start with run.sh, then inspect run.py and embeddings_extraction/etl/pipeline.py around the pipeline options and the reported temp_location and staging_location values. Reproduce the command with the documented environment and verify that the path validation errors are resolved and the pipeline submission behavior is clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python, shell
Domain
cloud, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.