[Feature Request] In-place materialize of base-referenced files (promote a shallow clone / branch to independent without deep_clone)

Summary

Add an operation to materialize a dataset's base_paths-referenced files into its own storage in place — copy the referenced data (and index) files local, rewrite the referencing DataFiles to base_id = None, and drop the now-unreferenced BasePaths. This is the in-place counterpart to deep_clone (which only produces a new dataset) and the missing remove half of base-path management (UpdateBases only adds). It is what lets a shallow clone — or, with #7263, a branch that adopted another branch's fragments — become self-contained so the source can be safely deleted (#7185).

Motivation

After a shallow_clone (or a future branch merge / rebase, #7263), a dataset/branch holds DataFiles whose base_id points into another base. To make it independent — so the source dataset/branch can be vacuumed or deleted without orphaning it (the hazard in #7514) — those files must become local. Today the only way is deep_clone, which copies to a new URI and refuses if a dataset already exists at the target (Dataset::deep_clone, rust/lance/src/dataset.rs). There is no way to re-home the base-referenced files of an existing dataset/branch in place, keeping its version/lineage chain.

Background: base references today

• Manifest.base_paths: HashMap<u32, BasePath> + DataFile.base_id: Option<u32> resolve a file relative to a registered base, or to the dataset root when base_id is None (rust/lance-table/src/format/manifest.rs, rust/lance-table/src/format/fragment.rs).
• Manifest::shallow_clone and Operation::UpdateBases add bases — UpdateBases is add-only (proto new_bases; apply in rust/lance/src/dataset/transaction.rs only inserts).
• compact_files re-homes only the fragments it happens to rewrite; DataReplacement swaps a single fragment's DataFile. Neither is a complete "materialize all base-referenced files and drop the bases" sweep.

So there is a shallow-clone path (reference) and a deep_clone path (full copy to a new dataset), but no in-place shallow → independent transition.

Proposed mechanism

materialize_bases(dataset):
  1. for each DataFile with base_id = Some(_):
       object-store copy the file into the dataset's own data dir
       (and index files under _indices/), preserving fragment ids + row-id meta
  2. commit DataReplacement (data) / re-commit CreateIndex with base_id = None (indices),
       rewriting the DataFiles to base_id = None  (rows + fragment ids unchanged)
  3. drop the now-unreferenced BasePaths
       (the remove half of UpdateBases — currently missing)

Preserving fragment ids and row-id meta keeps index fragment_bitmaps valid, so indexes don't need rebuilding (a base_id-carrying index can be re-homed by copying its files and re-committing the IndexMetadata with base_id = None). The result is a dataset with no base_paths, identical contents, on the same version chain — at which point the source it cloned from can be deleted (#7185) with no dangling references.

Related: #5249 (deep_clone, the new-dataset alternative), #4864 (the add-bases API; this is the remove half), #7263, #7185, #7514.

Start by reading rust/lance-table/src/format/manifest.rs and fragment.rs, then inspect Dataset::deep_clone and UpdateBases application in rust/lance/src/dataset/transaction.rs. Trace how DataReplacement, CreateIndex, base_id, and fragment metadata are handled. Done means all base-referenced data and index files are local, references have base_id = None, BasePaths are removed, and contents and lineage remain unchanged.