[to be discussed] Configure clean on spark to gracefully handle a large increase in uncleaned files.

### Task Description

**What needs to be done:**
For instant/duration based cleaning policy (KEEP_LATEST_COMMITS/KEEP_LATEST_BY_HOURS), when would like the implementation of clean planner/execution in spark to be optimized to meet the following requirement:

- If there are new files to be cleaned, then the next clean should be scheduled and executed in a bounded amount of time/memory usage, regardless of how many seconds/commits have elapsed since the prior ECTR (clean's earliest commit to retain). It is acceptable if the new `.clean` only makes "partial" progress and blocks the new ECTR, as long as the next clean plan "resumes" from where this previous clean completed.

**Why this task is needed:**

For context, we would like to handle the following scenarios that we have seen for our use cases
1. A dataset has an ingestion spark job which only writes bulk-insert instants to the latest partition. But after a few years, a second concurrent job does a clustering write. This causes the next clean planner attempt (in the ingestion job) to scan thousands of partitions in the dataset, since the prior ECTR doesn't exist or there have been many instants since it. We have seen increased runtimes and spark driver OOM failures - even if there are only a few files to actually "clean" in the clean plan.
2. An update-heavy dataset has had no cleans be attempted (due to a misconfiguration or orchestration issue). When clean runs again, it can OOM when creating the .clean/.clean.requested due to a high # files to clean.

Note that both incremental and non-incremental clean planner determine a list of partitions, and find all updated/replaced file groups before new/proposed ECTR. The difference is the non-incremental clean scans all partitions in the dataset, instead of only partitions referenced by instants since the latest ECTR.

**Suggested approach**

In our organizations internal HUDI build (for table version 6), we have partially minimized these issues by
- Block archival from archiving instants before clean's ECTR. And to ensure instants are eventually archived, we update clean to create an ["empty" clean plan once a day](https://github.com/apache/hudi/pull/11605/changes).This ensures that clean planer will incrementally clean since ECTR will always be present in active timeline. We can upstream this change, and maybe alter the implementation to not publish the empty clean daily but instead only if there are too many instants in active timeline.
-- Note that this does not optimize the case of clean planner needing to process many partitions (in case all instants since last ECTR have touched many partitions).
- Add a config to control number of instants to clean - for example if this is set to 3, then the new ECTR of clean planner will only "move" 3 instants ahead. Unfortunately this needs to be manually configured, since datasets may have different update/replace workloads. But internally this has proven to be a helpful "lever" for us to mitigate cases where a dataset suddenly has situation (2) with a large amount of data files updated.

### Task Type

Code improvement/refactoring

### Related Issues

**Parent feature issue:** (if applicable )
**Related issues:**
NOTE: Use `Relationships` button to add parent/blocking issues after issue is created.