apache / apache/beam

Add ability to specify gRPC server/worker ports

Open
#21,087 0 comments 0 reactions 0 assignees View on GitHub
beam-model improvement P3 runners universal
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

My development setup is all Docker-based and I work inside a Docker container for development. From this docker environment, i'd love to be able to [run a pipeline with a custom container image](https://beam.apache.org/documentation/runtime/environments/) for local testing. I've been able to launch a sister beam docker container from my main docker container, but am running into an issue with the ports.

It would be really helpful if I could set a value for the control_address and logging_endpoint of the GrpcServer for the portable runner worker.

The 'docker run' command passes in a control_address [here](https://github.com/apache/beam/blob/2aab1b04c2f5002527e0f2d25075b282feb7c054/sdks/python/apache_beam/runners/portability/fn_api_runner/worker_handlers.py#L767-L769) which ultimately comes from [here.](https://github.com/apache/beam/blob/2aab1b04c2f5002527e0f2d25075b282feb7c054/sdks/python/apache_beam/runners/portability/fn_api_runner/worker_handlers.py#L464) Similarly, the logging_endpoint passed in  [here](https://github.com/apache/beam/blob/2aab1b04c2f5002527e0f2d25075b282feb7c054/sdks/python/apache_beam/runners/portability/fn_api_runner/worker_handlers.py#L766) comes from [here](https://github.com/apache/beam/blob/2aab1b04c2f5002527e0f2d25075b282feb7c054/sdks/python/apache_beam/runners/portability/fn_api_runner/worker_handlers.py#L515).

This would be a pretty small change, just exposing some knob to configure those endpoints rather than having them chosen by the gRPC server. Thanks!!

Imported from Jira [BEAM-12580](https://issues.apache.org/jira/browse/BEAM-12580). Original Jira may contain additional context.
Reported by: mhoeger.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.