apache / apache/beam

Docker options in --environment_config

Open
#19,869 0 comments 0 reactions 0 assignees View on GitHub
bug harness P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

`I'm trying to mount a directory by providing additional arguments via –environment_config in the PipelineOptions:`

 

`pipeline_options = PipelineOptions([{color:#6a8759}"--runner=PortableRunner"{color}{color:#cc7832},{color} {color:#6a8759}"--job_endpoint=localhost:8099"{color}{color:#cc7832},{color} {color:#6a8759}"--environment_config=-v /tmp:/tmp benjamintan-docker-apache.bintray.io/beam/python3:latest"{color}]{color:#cc7832},{color} {color:#aa4926}pipeline_type_check{color}={color:#cc7832}True{color})`

 

However,  the command fails with the following:

 

 

`RuntimeError: Pipeline BeamApp-benjamintan-0916144446-839e633f_994659f0-7da9-412e-91e2-f32dd4f24b5c failed in state FAILED: java.io.IOException: Received exit code 125 for command 'docker run -d --mount type=bind,src=/home/benjamintan/.config/gcloud,dst=/root/.config/gcloud --network=host --env=DOCKER_MAC_CONTAINER=null -v /tmp:/tmp benjamintan-docker-apache.bintray.io/beam/python3:latest --id=7-1 --logging_endpoint=localhost:41835 --artifact_endpoint=localhost:40063 --provision_endpoint=localhost:39827 --control_endpoint=localhost:45355'. stderr: unknown flag: --idSee 'docker run --help'.`

 

However, if I were to copy and paste the `docker run ...` command, the command seems OK (no syntax errors)

 

This seems related to BEAM-5440. It isn't clear if there's a "right" way to pass in additional Docker run arguments.

Imported from Jira [BEAM-8239](https://issues.apache.org/jira/browse/BEAM-8239). Original Jira may contain additional context.
Reported by: bentanweihao.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.