Backup must be able to handle arbitrarily large cluster commit version advances.

Background: FDB Backup currently writes about 1 mutation log file to the backup per 20 seconds worth of commit versions. The backup log dispatch task creates one backup log range task for each file, under the assumption that there will be data in most 20 second periods. There are times when the cluster's commit version is advanced artificially, which can cause a burst of tasks to be created for several empty version ranges without any committed mutations and therefore also empty log files for each of those empty version ranges. For example, a master recovery advances the cluster commit version by 90 seconds, which will create at least 4 empty backup log ranges tasks and files.

The problem occurs when backup is running while the cluster commit version is advanced by a very large amount. There are at least two sources of this:

- A restore of a backup into the cluster
- A DR switch event

The primary reason is that restored data or incoming data from DR may reference a cluster commit version, such as when using version stamped keys, and this version referenced by data should never be in the future. Therefore, in both cases the destination cluster is advanced to at least the version of the source data.

A backup that is running during such an advancement can suddenly have to create tasks for and then write files for many millions of empty version ranges. The backup log dispatch task cannot handle this and runs out of memory, which is actually good because if it could handle it then millions of empty files would be written to the backup.

The log range dispatch needs to react to large version deltas by checking for a large empty range and handling it efficiently with a single small log file written to the backup that covers the entire range. Or at the very least an exception should be thrown if the log range dispatch detects a version delta over some threshold to prevent it from crashing (but probably far smaller than the crash threshold).

@jzhou77 How would the partitioned log writer handle this type of version advancement?

Start by tracing the backup log dispatch task and reviewing how the partitioned log writer handles large version advances. Reproduce or inspect behavior for a large empty commit-version range; done means the dispatch avoids millions of tasks and files by handling the range efficiently or rejecting it safely before running out of memory.

Written by the indexing model from the issue text.