apache / apache/beam

"Socket closed" Spurious GRPC errors in Flink/Spark runner log output

Open
#20,149 0 comments 0 reactions 0 assignees View on GitHub
flink P3 portability runners spark sub-task
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

We often see "Socket closed" errors on job shutdown, even though the pipeline has finished successfully. They are misleading and especially annoying at scale.

ERROR:root:Failed to read inputs in the data plane.
...
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Socket closed"
debug_error_string = "{"created":"@1578597616.309419460","description":"Error received from peer ipv6:[::1]:37211","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Socket closed","grpc_status":14}"

Imported from Jira [BEAM-9082](https://issues.apache.org/jira/browse/BEAM-9082). Original Jira may contain additional context.
Reported by: ibzib.
Subtask of issue #20148

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.