Flink Native HA on Kubernetes is not supported
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 581
- Forks
- 153
- PR merge metrics
- No merged PRs in 30d
Description
I tried out the [native HA on Kubernetes](https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/ha/kubernetes_ha/) using the operator.
Here is the general synopsis:
- If I start a fresh new deployment, the deployment succeeds and the application comes up perfectly
- If I try to perform a deployment (update) of an existing application, then the deployment fails.
The deployment (update) of an existing application goes through the following:
- New job manager and task manager pods for the new deployment are created
- The old job is canceled on the old pods
- The new job manager tries to start the job and prints "Submitting Job with JobId=<>", but fails repeatedly with "The connection was unexpectedly closed by the client."
- The old job manager eventually starts a job with a new job id
- The new pods are destroyed and the old cluster continues running with the old code
Has anyone been successful in using Native K8s HA with Flink with this FlinkK8sOperator?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Native HA on Kubernetes documentation and reproduce both a fresh deployment and an update of an existing application using the FlinkK8sOperator. Inspect the new and old job manager logs around job submission and the connection closure; done means an update succeeds and the new pods run the new code without reverting to the old cluster.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100