lyft / lyft/flinkk8soperator

support for checkpoint-based updates

Open
#197 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
581
Forks
153
PR merge metrics
No merged PRs in 30d

Description

Our Flink job deploys rely heavily on checkpoints since our savepoints take around 30 -45 minutes to write and read back in on the new job.

It appears that enabling of `savepointDisabled` gets us part of the way there and that there exists mechanisms for relying on checkpoints to recover a failing job.

We set `ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION` from within our job and we'd really like to be able to rely only on checkpoints to update jobs.

The way I envision still supporting checkpoints, say for when we need to change parallelism, would be to submit a new job with `savepointDisabled` disabled such that the next job update would use savepoints.

I'm happy to work on this if a PR would be accepted.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing how the operator handles savepointDisabled, ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION, and job updates. Compare the existing savepoint flow with the requested checkpoint-based recovery behavior, including the proposed parallelism-change case. Done means updates can rely on checkpoints by default while still using savepoints when explicitly enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.