Failure deployment doesn't change the state in flink operator
- Dominant language
- Go
- Stars
- 581
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
According to the states, if dual mode, either failure in `ClusterStarting` or `SubmittingJob` will leads to the `RollingBackJob` mode. However, when I tested, I noticed that
1. In `ClusterStaring` state, the deployment failed (usually due to the image not exists or the sidecar is not injected properly that we are missing some packages), the flink app will stuck in `ClusterStarting` and not really goes into the `RollingBackJob` state.
2. In `SubmittingJob`, sometime bad beam python code written and then it will also stuck in `SubmittingJob` state, and I can see that it keeps trying to resubmit the job (and showing error log in flink operator) instead of changing to `RollingBackJob` state as mentioned in the [doc](https://github.com/lyft/flinkk8soperator/blob/master/docs/state_machine.md)
Wondering if I am missing some configuration which leads to this issue. Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/state_machine.md and compare the documented ClusterStarting and SubmittingJob failure transitions with the observed behavior. Reproduce a failed deployment using the image or sidecar case, then a bad Beam Python submission, and trace the operator logs and state changes. Done means both failures enter RollingBackJob instead of remaining in their current states or repeatedly resubmitting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100