[Discussion/Proposal WIP] Add support to soft delete partitions

### Feature Description

Currently, HUDI delete_partition API will replace all files in the partition, and HUDI clean table service will delete these files and entries from MDT once the instant is eligible for clean.

We want to add support for being able to "soft delete" a partition, and later have all files and references in MDT/indexes be actually deleted. The idea is that in between this period (between the "soft" delete and full cleanup)
- users can "recover" the data if needed
- readers/queries should not return any data from these partitions
- table service planning should ignore these partitions
- writers attempting to upsert/replace records should fail

### User Experience

**How users will use this feature:**
- Configuration changes needed
- API changes
- Usage examples

### Hudi RFC Requirements

**RFC PR link:** (if applicable)

**Why RFC is/isn't needed:**
- Does this change public interfaces/APIs? (Yes/No)
- Does this change storage format? (Yes/No)
- Justification:

No files, tests, or entry points are named; begin by mapping Hudi's delete_partition API, clean table service, MDT, indexes, readers, and writers. Done would require an agreed design and implementation for recoverable soft deletion, hidden partition data, ignored table-service planning, rejected upserts/replacements, and later full cleanup.

Written by the indexing model from the issue text.