Flink's job server container attempts to invoke docker
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
I want to run a Java pipeline with `--runner PortableRunner`, so I do the following:
* Run the Flink's job server with:
```
docker run --rm --net=host apache/beam_flink1.13_job_server:latest
```
* Run the pipeline with:
```
mvn exec:java -Dexec.mainClass=org.foo.bar.MyMainClass -Pportable-runner -Dexec.args="--runner=PortableRunner
--jobEndpoint=localhost:8099"
```
After this I get the error:
```
java.lang.Exception: The user defined 'open()' method caused an exception: java.io.IOException: Cannot
run program "docker": error=2, No such file or directory
at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:513)
at
org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:360)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:779)
at
org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
```
What I did to workaround this is to use the `gradlew` script from the Git repository like this:
```
./gradlew :runners:flink:1.13:job-server:runShadow
```
and it works without problem.
Imported from Jira [BEAM-12642](https://issues.apache.org/jira/browse/BEAM-12642). Original Jira may contain additional context.
Reported by: itaranto.
Contributor guide
Research direction
Start by reproducing the Docker image command and the Java PortableRunner invocation shown in the report, then compare them with :runners:flink:1.13:job-server:runShadow. Trace why the container attempts to invoke docker and verify the job server can run the pipeline without requiring a docker executable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100