apache / apache/beam

Spark runner should not used STOPPED state as a terminal state

Open
#19,919 0 comments 0 reactions 0 assignees View on GitHub
bug P3 runners spark
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Over at BEAM-8539 we're trying to iron out the definitions of the job state enums.  One thing that is clear is that STOPPED is not supposed to be a terminal state, though since it was poorly documented it slipped in a few places.

There are two places in the `SparkPipelineResult` where STOPPED seems to be used as a terminal event.   These should either be DONE, FAILED, or CANCELLED.

Here's one example:

[https://github.com/apache/beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L133](https://github.com/apache/beam/blob/master/runners/spark/src/main/java/org/apache/beam/runners/spark/SparkPipelineResult.java#L133)

 

 

Imported from Jira [BEAM-8552](https://issues.apache.org/jira/browse/BEAM-8552). Original Jira may contain additional context.
Reported by: chadrik.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.