Worker endpoints are unset in StartWorkerRequest.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
User reported that worker endpoints are unset when attempting to start a worker in a worker pool using Kubernetes. https://stackoverflow.com/questions/66498209/how-to-configure-beam-python-sdk-with-spark-in-a-kubernetes-environment
Starting worker with command ['/opt/apache/beam/boot', '\--id=1-1', '\--logging_endpoint=', '\--artifact_endpoint=', '\--provision_endpoint=', '\--control_endpoint=']
These endpoints are expected to be set to use localhost and some random allocated port. The server is started here: https://github.com/apache/beam/blob/b101064f18e71adb81c8fe6ba7beae53e9b13b08/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L672-L697
And the StartWorkerRequest is constructed here: https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ExternalEnvironmentFactory.java
Imported from Jira [BEAM-11957](https://issues.apache.org/jira/browse/BEAM-11957). Original Jira may contain additional context.
Reported by: ibzib.
Contributor guide
Assessment
This issue has not been assessed yet.