[Variant] Remove Variant UTF-16 output rewriting

### What is the problem the feature request solves?

PR #5407 must bridge two incompatible Variant object-key orders:

- Parquet Variant and Arrow use unsigned lexicographic UTF-8 byte order.
- Spark versions predating SPARK-58949 built and searched Variant objects using Java UTF-16 order, so existing Spark-written Parquet files can contain that legacy order.

Comet currently preserves correctness by recursively examining and sometimes re-encoding Variant values at the native Parquet normalization boundary:

- [Normalized output is converted to Spark UTF-16 order](https://github.com/apache/datafusion-comet/blob/23bfd83b8ffc1071b487a90d8dff26019bb2f1bf/native/core/src/parquet/cast_column/variant.rs#L38-L87).
- [`SparkMetadataBuilder` and the recursive compatibility scan implement both orderings](https://github.com/apache/datafusion-comet/blob/23bfd83b8ffc1071b487a90d8dff26019bb2f1bf/native/core/src/parquet/cast_column/variant.rs#L654-L762).
- [Partially shredded residual values are temporarily converted to Arrow UTF-8 order before unshredding](https://github.com/apache/datafusion-comet/blob/23bfd83b8ffc1071b487a90d8dff26019bb2f1bf/native/core/src/parquet/cast_column/variant.rs#L541-L582).
- [Shredded values are rebuilt for Spark compatibility](https://github.com/apache/datafusion-comet/blob/23bfd83b8ffc1071b487a90d8dff26019bb2f1bf/native/core/src/parquet/cast_column/variant.rs#L1269-L1348).

This compatibility layer is necessary today, but it adds recursive per-row traversal, allocations when re-encoding is required, and substantial code in the scan path.

This is a focused follow-up to #5438 and #5407.

### Describe the potential solution

After every Spark 4 profile supported by Comet includes SPARK-58949 or equivalent behavior:

1. Export Arrow-rs canonical Variant bytes directly instead of converting normalized output to Spark UTF-16 order.
2. Remove `SparkMetadataBuilder`, `VariantObjectKeyOrder::SparkUtf16`, and the recursive output compatibility/re-encoding path.
3. Preserve only the narrow input-side conversion needed for legacy Spark Parquet files whose residual `value` is UTF-16 ordered. Prefer upstream Arrow-rs support; otherwise document when that compatibility path can be retired.
4. Benchmark whole-value Variant projection before and after the change so the removed traversal has measured impact.

Definition of done:

- Canonical Parquet Variant files and legacy Spark-written files remain readable.
- Top-level, nested, and partially shredded objects with Unicode keys round-trip correctly.
- Spark field lookup works for objects above its binary-search threshold.
- Native output no longer performs a recursive UTF-16 rewrite.
- Obsolete compatibility code and tests are removed, or reduced to a documented legacy-input path.
- Spark 3.x behavior remains unchanged.

### Additional context

- [SPARK-58949](https://issues.apache.org/jira/browse/SPARK-58949) is implemented by [commit `2a61a79b`](https://github.com/apache/spark/commit/2a61a79bbb73b46cde33e81b9916f6d47e76efe2) on Spark `master` and [commit `789106b0`](https://github.com/apache/spark/commit/789106b01b72aa7d1c6f424eaa077fd592876a72) on `branch-4.x`. Spark now writes and searches canonical unsigned UTF-8 order and retries lookup using legacy UTF-16 order. [PR #58239](https://github.com/apache/spark/pull/58239) appears closed rather than merged because the change was landed as direct commits.
- [parquet-java issue #3735](https://github.com/apache/parquet-java/issues/3735) tracks the same ordering problem.
- [parquet-java PR #3746](https://github.com/apache/parquet-java/pull/3746) is open and implements canonical UTF-8 write/search plus legacy UTF-16 lookup fallback.