consolidate bitmap segments and unindexed data on optimize

What this PR does

unify its segment build path with the common scalar flow, teach optimize_indices to consolidate Bitmap segments and unindexed data in one pass with old-data filtering, and delete the legacy distributed shard machinery that main has already soft-broken.

1. Remove the legacy shard path

main already soft-broke the only entry point (merge_index_metadata for Bitmap returns "no longer supports merge_index_metadata"), so the shard machinery was unreachable dead code. Deleted ~450 lines: part_* shard files, train_bitmap_shard, K-way heap merge, merge_index_files.

• shard_id is deprecated: ignored with a warning.
• index_uuid is rejected for fragment-scoped BTree/Bitmap builds; segment UUIDs are generated by Lance.

2. optimize with old-data filtering

• optimize_indices now consolidates N Bitmap segments + unindexed fragments in one pass (previously: full rebuild). A single segment still takes the streaming BitmapIndex::update path.
• New OldIndexDataFilter::retain_row_addrs drops rows from retired fragments; merge_segments intersects segment coverage with live fragments instead of blindly unioning.

3. Unify the segment build path

Bitmap segment builds now go through the common build_scalar_index flow; the Bitmap-only special case in create.rs is deleted, and uses_segment_commit_path covers Bitmap uniformly (including IndexType::Scalar + bitmap params).

Start by tracing optimize_indices, merge_segments, and the common build_scalar_index flow, then read create.rs and the OldIndexDataFilter and BitmapIndex::update paths. Done means the legacy shard machinery is removed, Bitmap and unindexed data consolidate with retired-row filtering, and segment commit behavior is uniform as described.