apache / apache/beam

[Bug]: Beam Pipeline does not halt execution while running on non local Spark Cluster

Open
#23,932 10 comments 0 reactions 0 assignees View on GitHub
bug harness P3 portability python spark
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

I have deployed a Spark v3.1.2 cluster on kubernetes. My beam job server and beam sdk container are running on 2 separate linux virtual machines. The following keeps executing and does not stop

`op = PipelineOptions([
"--runner=PortableRunner",
"--job_endpoint=localhost:8099",
"--environment_type=EXTERNAL",
"--environment_config=vm2-hostname::50000",
"--artifact_endpoint=localhost:8098"
]
)`

`with beam.Pipeline(options=op) as p:
p | beam.Create([1, 2, 3, 10]) | beam.Map(lambda x: x+1) | beam.Map(print)`

The docker logs for the sdk container show the following error"

_Starting worker with command ['/opt/apache/beam/boot', '--id=1-1', '--logging_endpoint=localhost:43541', '--artifact_endpoint=localhost:44461', '--provision_endpoint=localhost:43225', '--control_endpoint=localhost:35475']
E1101 22:11:16.787804592 22 fork_posix.cc:76] Other threads are currently calling into gRPC, skipping fork() handlers
2022/11/01 22:13:16 Failed to obtain provisioning information: failed to dial server at localhost:43225
caused by:
context deadline exceeded_

### Issue Priority

Priority: 3

### Issue Component

Component: sdk-py-harness

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.