[Improve][Connector-V2][JDBC] Excessive scanning and split-memory growth when reading large tables

### Search before asking

I reviewed #10288 (database-side sampled balanced sharding) and #10596 (allow disabling sampling). This issue describes the combined impact of sampling scans and split-metadata growth during large JDBC reads, including chunk generation, assignment, reader queues, and checkpoint state. Please link or consolidate this issue if the community prefers one tracking issue.

### Description

Large JDBC reads can perform substantial extra database and network work during split planning, while retaining very large collections of sample values and split metadata in JVM memory.

There are two connected problems: the sampling query can consume the entire split-key result even at a low sampling rate, and the resulting splits are generated and queued in full. Reducing retained sample values does not address the split backlog; disabling sampling does not remove the eager split collections. At sufficiently large scales, this creates long planning times, heavy GC pressure, and a risk of JVM out-of-memory failures.

This issue documents the problem and its effects. The implementation approach and scope are open for contributors to determine.

#### Current upstream behavior

The following findings were checked against `dev` commit [`ce4cde54c8abc7a6a9e3831110dbab029512ef7b`](https://github.com/apache/seatunnel/commit/ce4cde54c8abc7a6a9e3831110dbab029512ef7b). They are source-level observations, not the result of a new large-scale benchmark or an attached heap dump.

1. **The default sampling path consumes the entire query result before retaining every Nth row.** `JdbcDialect.sampleDataFromColumn()` executes `SELECT split_column FROM table` (or from the configured subquery), iterates the complete ResultSet, retains every `samplingRate`-th value in an `ArrayList`, converts it to an array, and sorts it. Thus increasing the inverse sampling rate reduces retained samples but not the rows traversed and returned through JDBC. A covering index may change the physical access path, but it does not make this a bounded sample query. [Default implementation](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/JdbcDialect.java#L385), [Oracle override](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/oracle/OracleDialect.java#L333).

2. **Chunk and split collections are eagerly materialized.** `DynamicChunkSplitter.createDynamicSplits()` obtains the complete `List` and builds a second `List`. Both sampled and iterative uneven splitting accumulate chunk ranges. This leaves memory proportional to the number of generated boundaries/splits even if the sampling query becomes cheaper. [Conversion](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/DynamicChunkSplitter.java#L81), [sampled and iterative splitting](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/DynamicChunkSplitter.java#L632).

3. **Assignment moves the backlog to readers rather than bounding it.** `JdbcSourceSplitEnumerator.run()` generates all splits for a table, groups them into pending lists, and assigns each reader its complete list. `handleSplitRequest()` currently throws unsupported-operation. `JdbcSourceReader.addSplits()` appends all received splits to an uncapped deque, and `snapshotState()` copies that deque. The enumerator state also contains its pending split lists. [Enumerator](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceSplitEnumerator.java#L73), [Reader](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceReader.java#L38), [state](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/state/JdbcSourceState.java#L32).

4. **Disabling sampling is already supported, but does not solve the full resource problem.** With `split.allow-sampling=false`, the uneven-data path uses iterative chunk-boundary queries and still builds the full chunk list. With sampling enabled, this path is chosen when the distribution heuristic considers the key uneven and the estimated split count exceeds `split.sample-sharding.threshold`. [Strategy selection](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/DynamicChunkSplitter.java#L455).

#### Illustrative scale

For a table with 100 billion rows, an **explicitly configured** `split.size=100000`, and inverse sampling rate 1000:

- The estimated split count is 1 million; the actual count depends on key distribution and boundary deduplication.
- The current full-result sampling path can retain approximately 100 million sample values and then sort them.
- A plan with approximately 1 million distinct boundaries produces correspondingly large chunk/split lists, assignment payloads, reader queues, and recoverable pending state.

These are arithmetic scaling examples, not measured object sizes or a claim that every such job necessarily OOMs. Actual memory depends on types, heap sizing, serialization, and consumption speed. The inspected upstream default `split.size` is **8096**, not 100000. [Defaults](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/config/JdbcSourceOptions.java#L49).

#### Resource impact and configuration tradeoffs

- **Database and network load before processing:** sampling can scan a complete table or index and transfer the full split-key result through JDBC, even when only a small fraction is retained. The iterative alternative issues repeated boundary queries whose total count and scan cost grow with the required splits and depend on the query plan.
- **Coordinator memory and GC:** sample collection, array conversion/sorting, chunk ranges, and split objects can all contribute to planning-time memory pressure. Collections remain proportional to the table's sample or split count rather than the work currently being processed.
- **Reader memory and checkpoint size:** assigning splits does not mean they have been consumed. Slow readers retain their backlog, and checkpoint snapshots include the remaining split metadata. Multiple tables can add further queued work before earlier tables have finished.
- **Existing knobs have different effects:** increasing `split.inverse-sampling.rate` reduces retained samples but leaves the full ResultSet traversal; changing `fetch_size` affects fetching rather than the total result; disabling sampling changes the boundary-query strategy but leaves eager split materialization.
- **Larger splits have their own costs:** increasing `split.size` can reduce the number of split objects, but increases the work assigned to each split and can worsen long-tail behavior on skewed data. The current reader processes a whole split while holding the checkpoint lock, so larger splits can also increase checkpoint waiting and the amount of work repeated after failure. [Reader execution](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSourceReader.java#L62).
- **Sampling is not the only planning query:** the inspected Oracle row-count path can execute `ANALYZE ... COMPUTE STATISTICS` unless `skip_analyze` is set, or use exact counting for query-based reads. These operations can add database work independently of the sampling query. [Oracle row-count path](https://github.com/apache/seatunnel/blob/ce4cde54c8abc7a6a9e3831110dbab029512ef7b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/internal/dialect/oracle/OracleDialect.java#L224).

#### Expected behavior

Large JDBC reads should be able to complete without excessive extra planning scans or JVM memory consumption caused by retaining the full set of samples and splits. This expectation includes multi-table workloads and slow readers, not only a single table with immediately available consumers.

Data coverage, configured query/filter semantics, NULL and duplicate-key handling, and existing checkpoint/recovery guarantees must remain correct. The problem is especially important when key values are sparse or skewed, because reducing the number of splits can create a few disproportionately expensive reads.

Community investigation, alternative approaches, and evidence from affected workloads are welcome. This issue does not prescribe an algorithm, architecture, change size, or implementation sequence.

### Usage Scenario

Large JDBC batch reads with billions to hundreds of billions of rows, uneven keys, multiple tables, and source databases where extra planning scans are expensive. The affected resources include the source database, network, Coordinator JVM, and Reader JVMs. CDC snapshot splitting is related, but has a separate lifecycle; the source evidence in this issue concerns the JDBC connector.

### Related issues

- #10288: database-side sampled balanced sharding. This issue also documents split-metadata growth through generation, assignment, readers, and checkpoint state.
- #10596: disabling sampling. The current `split.allow-sampling` option is acknowledged; this is not another request for the switch.

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

Opening this to describe the problem and invite contributors to investigate and propose a solution.

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct).

Start by reading JdbcDialect.java, DynamicChunkSplitter.java, JdbcSourceSplitEnumerator.java, JdbcSourceReader.java, and JdbcSourceState.java, following sampling, split generation, assignment, and checkpoint flows. Define and validate an approach that bounds planning and queued metadata while preserving coverage, query semantics, NULL and duplicate handling, and checkpoint recovery for slow readers.

Written by the indexing model from the issue text.