GoogleCloudPlatform / GoogleCloudPlatform/realtime-embeddings-matching
Error in running the command bash run.sh
- Dominant language
- Python
- Stars
- 129
- Forks
- 29
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am getting the following error while running the command bash run.sh
Cleaning working and output directories...
CommandException: "rm" command does not support "file://" URLs. Did you mean to use a gs:// URL?
CommandException: "rm" command does not support "file://" URLs. Did you mean to use a gs:// URL?
Running the Dataflow job...
********************************************************************************
Python 2 is deprecated. Upgrade to Python 3 as soon as possible.
See https://cloud.google.com/python/docs/python2-sunset
To suppress this warning, create an empty ~/.cloudshell/no-python-warning file.
The command will automatically proceed in seconds or on any key.
********************************************************************************
Traceback (most recent call last):
File "run.py", line 69, in
main()
File "run.py", line 64, in main
pipeline.run(pipeline_options, known_args)
File "/home/shahmansi78/realtime-embeddings-matching/text-semantic-search/embeddings_extraction/etl/pipeline.py", line 103, in run
pipeline = beam.Pipeline(options=pipeline_options)
File "/home/shahmansi78/.local/lib/python2.7/site-packages/apache_beam/pipeline.py", line 149, in __init__
'Pipeline has validations errors: \n' + '\n'.join(errors))
ValueError: Pipeline has validations errors:
Invalid GCS path (wordembedding/wikipedia/dataflow/temp), given for the option: temp_location.
Invalid GCS path (wordembedding/wikipedia/dataflow/staging), given for the option: staging_location.
Dataflow job submitted successfully!
Contributor guide
Research direction
Start with run.sh, then inspect run.py and embeddings_extraction/etl/pipeline.py around the pipeline options and the reported temp_location and staging_location values. Reproduce the command with the documented environment and verify that the path validation errors are resolved and the pipeline submission behavior is clear.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python, shell
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100