[Feature](lance) Support global BM25 statistics for distributed FTS

## Parent

- Tracked by #66493
- Part of #66340
- Doris integration draft: #67289
- Lance tracking issue: https://github.com/lance-format/lance/issues/8937

## Motivation

Doris executes Lance full-text search in a distributed architecture. The FE plans one query and assigns different Lance fragments or FTS index segments to multiple BEs.

If each BE builds BM25 statistics from only its local segments, its document count, total token count, average document length, and query-term document frequencies can differ from other workers. Those worker-local scores are not comparable, and merging local Top-K results cannot recover the ranking produced by one search over the complete index.

The FE Java layer therefore needs to collect query-bound, corpus-wide BM25 statistics for one pinned Lance dataset snapshot. The FE stores the resulting opaque payload in `TFullTextSearchParams.global_statistics`. That field is delivered directly to every BE through the existing `TLanceScanParams.external_search_request` Thrift scan plan; this design does not introduce a separate broadcast or coordination channel.

```mermaid
flowchart TB
subgraph FE["Doris FE - Java query planning"]
Q["FTS query"] --> P["Pin Lance dataset version
resolve complete FTS segment set"]
P --> J["Lance Java API
collect global BM25 statistics"]
J --> T["Set TFullTextSearchParams.global_statistics"]
M["Merge comparable BE Top-K results"]
end

subgraph LANCE["One logical Lance FTS index"]
S1["FTS segment 1"]
S2["FTS segment 2"]
S3["FTS segment 3"]
end

T --> THRIFT["Existing Thrift scan-plan delivery"]

subgraph BES["Doris BE cluster"]
BE1["BE 1
consume global statistics
search assigned segments"]
BE2["BE 2
consume global statistics
search assigned segments"]
BE3["BE 3
consume global statistics
search assigned segments"]
end

S1 --> J
S2 --> J
S3 --> J
THRIFT --> BE1
THRIFT --> BE2
THRIFT --> BE3
BE1 --> M
BE2 --> M
BE3 --> M
```

## V1 scope

The first version deliberately represents one BM25 corpus:

- one dataset version;
- one logical FTS index, indexed column, and document granularity;
- the exact committed segment UUID set for that corpus;
- corpus document count, total token count, and document frequency for every prepared term;
- the prepared vocabulary and original token positions for every scoring leaf, including fuzzy expansion.

Every scoring leaf must resolve to the same corpus. Cross-column or otherwise cross-corpus queries must be rejected.

The protobuf is an opaque payload within a trusted planning flow. The FE must attach it only to the same query used to produce it. Consumers validate the schema and dataset/index metadata, but V1 does not attempt to reconstruct and independently prove the original query identity.

## Existing Doris integration point

PR #67289 reserves `TFullTextSearchParams.global_statistics` as an optional opaque binary field. The FE creates the ScanNode-level `external_search_request`, and the BE receives the same request from `TFileScanRangeParams.lance_scan_params`.

The BE currently returns an unsupported error when `global_statistics` is present because the lance-c consumer API is not available yet.

## Upstream Lance work

- lance-format/lance#8938 defines the versioned protobuf wire contract.
- lance-format/lance#8822 implements global-statistics collection in Rust and exposes it through JNI and the Java API as an opaque protobuf payload. It depends on lance-format/lance#8938.

These PRs provide the FE-side producer. A follow-up Rust/lance-c consumer is still required before the Doris BE can apply the payload to its assigned segment subset.

## Remaining work

1. Merge the Lance protobuf and producer PRs.
2. Add the Rust/lance-c consumer that validates the payload and installs the prepared BM25 context before FTS execution.
3. Make the Doris FE populate `TFullTextSearchParams.global_statistics` through the Java API.
4. Make each Doris BE consume the field through lance-c for its assigned segments.
5. Define the behavior for unindexed fragments and mixed indexed/unindexed execution.

## Completion criteria

- Distributed FTS ranking matches a single full-index execution over the same dataset version and committed segment set.
- Regression tests cover multiple segments where worker-local and global BM25 statistics produce different rankings.
- Fuzzy queries replay the same globally prepared vocabulary and token positions.
- Unsupported cross-corpus queries fail with a clear error.
- EXPLAIN or runtime profiles show whether FE-provided global BM25 statistics are used.

Start at the Doris FE ScanNode path that creates external_search_request and sets TFullTextSearchParams.global_statistics, then inspect the BE lance-c unsupported path. Review Lance PRs #8938 and #8822 for the payload contract and producer API. Done means distributed ranking matches full-index execution, regression coverage includes fuzzy and cross-corpus cases, and usage is visible in EXPLAIN or runtime profiles.

Written by the indexing model from the issue text.