[dbsp] Keeping two checkpoints around is wasteful

With FT and periodic checkpointing enabled, the pipeline keeps two checkpoints in steady state. The second checkpoint isn't strictly needed and can be safely GC'd once the new checkpoint is full completed. If my understanding is correct, we only keep it around because it simplifies the GC algorithm: we compute the list of unused files by diffing the file lists of the two checkpoints.

This extra checkpoint can increase pipeline's storage significantly. I am looking at a customer pipeline that is processing a series of expensive transactions, causing checkpoints to get delayed by 30 minutes. Initially a fresh checkpoint costs nothing in terms of storage since it shares all its files with the live pipeline, but over time, the pipeline diverges from the checkpoint, especially under heavy load. A single checkpoint doubles pipeline's storage footprint in the worst case. Two checkpoints triple it.

This is not a correctness issue, but I am labeling it as a bug since it causes pipelines to crash despite plenty of provisioned storage.

Related issue: #7032