Python SDK not creating Flink checkpoints
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
I have been trying to get checkpointing to work with Apache Flink and Beam for the Python SDK, but without any success. I read a ton of documentation on how to get this working, but couldn't make any progress, so I have to assume that this is a bug. If not, then we need to at least fix the documentation (fundamentally).
The "bug" is reproducible with both the PortableRunner and the FlinkRunner **** uber JAR. I cannot really test the FlinkRunner without uber JARs, because I am submitting to a remote cluster.
The flink cluster is configured with:
```
state.checkpoint-storage: "filesystem"
state.checkpoints.dir: "file:///foo/bar/cp"
state.savepoints.dir:
"file:///foo/bar/sp"
execution.checkpointing.interval: "60s"
execution.checkpointing.externalized-checkpoint-retention:
"DELETE_ON_CANCELLATION"
```
(`{}/foo/bar{`} is a shared network mount)
When I submit a job, all I'm seeing in the Flink Web UI under job configuration is
```
Execution mode: PIPELINED
Max. number of execution retries: Cluster level default restart strategy
Job
parallelism: 120
Object reuse mode: false
```
"User Configuration" is empty and no checkpoints are created (both the Checkpoints tab and the checkpoints folder remain empty).
I tried setting
```
checkpointing_interval=30000,
externalized_checkpoints_enabled=True,
```
in my Beam submission config, but the result is the same.
When I try the FlinkRunner with `{}flink_submit_uber_jar=True{`}, it's the same again, but this time I also get the following warning and the job starts with a parallelism of 1 (I guess that's another bug):
```
WARNING:apache_beam.options.pipeline_options:Discarding invalid overrides: {'checkpointing_interval':
30000, 'externalized_
checkpoints_enabled': True, 'parallelism': 120}
```
Imported from Jira [BEAM-13675](https://issues.apache.org/jira/browse/BEAM-13675). Original Jira may contain additional context.
Reported by: phoerious.
Contributor guide
Assessment
This issue has not been assessed yet.