[Tracking] PageIO read-ahead, I/O coalescing, and cache block hole filling

## Summary

This issue tracks four reviewable PRs for PageIO read-ahead, range coalescing, and cache-block hole filling. Current heads: #67292 [270a7dadbb5](https://github.com/apache/doris/pull/67292/commits/270a7dadbb529b68c4f920cd3f3eca5b845b2cff), validation/follow-ups #67579 [d5c427fe337](https://github.com/apache/doris/pull/67579/commits/d5c427fe3372e6c65c8f2041dae6a3ee91d3f0d2), and infrastructure #67612 [cdec0499937](https://github.com/apache/doris/pull/67612/commits/cdec0499937f782daf86d919fb1cb87674c67db3). The shared master base remains `a6751eb3ac6b4667c3291544c5cfb1fae48e1556`.

The 2026-09-14 update synchronizes planning timers, page-driven scan advancement and object-owned operation cleanup. #67292 has **17 component commits**; its first five are exactly #67612. Complete-block submission, zero-copy handoff and low-priority admission are one component. #67579 keeps 28 chronological commits and records the follow-up mappings. All three branches are pushed; synchronization does not mean merge completion.

Regular scans now select candidate physical pages once after pruning and advance windows on data-page reads, including Page Cache hits. Eager/lazy initial windows are planned together. Exact-row point and dependent nested-column reads retain grouped page preparation. Existing window sizes, IO limits and hole-fill behavior remain unchanged.

TopN/global delayed-materialization second-stage integration remains outside the current PRs. The rowid fetcher, materialization operator, RPC Profile field and TopN-specific test files match the shared master base. Reusable exact-row page helpers remain in the point-query component.

## Motivation

Storage-compute separated queries pay both object-store request latency and read amplification:

- Data pages normally target 64 KiB before compression and can be around 20–30 KiB after compression, while the file-cache block is normally 1 MiB. Fetching a full block for a sparse page read transfers substantially more data than the query needs.
- Small scanner batches limit how many pages become visible together. Loading one page at a time leaves object-store reads serial even when later pages or other columns could be read concurrently.
- Exact-range reads and read-ahead need a cache population path: useful complete blocks can be written immediately, while partial blocks need background hole filling before entering the existing fixed-block asynchronous cache writer.

The intended result is bounded concurrent page reads, controlled I/O coalescing, and best-effort cache population. Performance improvements and safe defaults must be validated with workloads; this issue does not claim measured speedups.

## End-to-end architecture

```mermaid
flowchart TD
Regular["Regular segment scans after pruning"] --> Windows["ColumnReadAhead: select candidate pages once
advance windows on physical page reads"]
Windows --> Plan
Point["Point-query data pages and shared exact-row helpers"] --> Plan
Plan["FileRangePlanner: coalesce ranges and optionally complete boundary blocks"] --> Read
Read["FileRangeReadScheduler: query/BE byte admission; one range per thread-pool task"] --> Cache["CachedRemoteFileReader: exact reads without foreground cache writes"]
Cache --> Consume["PageIO consumes slices of completed ranges"]
Consume --> Origin{"Range read remote bytes?"}
Origin -- No --> Skip["Reuse existing data; skip duplicate writeback"]
Origin -- Yes --> Writeback["RangeCacheWriteback: split consumed ranges"]
Writeback --> Full["Complete cache blocks"]
Writeback --> Partial["Partial cache blocks"]
Partial --> Reuse{"Inflight buffer or cached/downloading block?"}
Reuse -- Yes --> Skip
Reuse -- No --> Queue["PartialBlockWritebackManager queue: merge queued fragments"]
Queue --> Eligible{"Aggregation deadline elapsed
and cache writer has capacity?"}
Eligible -- No --> Wait["Wait on condition variable
keep task queued and mergeable"]
Wait --> Eligible
Eligible -- Yes --> Fill["Block worker: recheck cache; complement and merge holes"]
Fill --> Count{"Merged hole ranges"}
Count -- Zero --> Full
Count -- One --> Inline["Read on the block worker"]
Count -- Multiple --> Pool["Shared HoleFillRemoteReadPool: parallel GETs"]
Pool --> Join["Join all accepted reads"]
Inline --> Result{"All reads succeeded?"}
Join --> Result
Result -- Yes --> Full
Result -- No --> Drop["Drop cache-only work"]
Full --> Persist["Existing AsyncCacheWriteManager"]
```

## Delivery plan

Each PR includes its own component tests. Open branches share the master base recorded above; later PRs follow the dependency order below. The checkboxes track **merged**, not merely opened, PRs.

```mermaid
flowchart LR
Infra["PR 1: range I/O infrastructure"] --> Hole["PR 2: cache writeback and hole filling"]
Hole --> Scan["PR 3: regular segment page read-ahead"]
Scan --> Exact["PR 4: point-query page read-ahead"]
```

- [ ] **PR 1 — File-range I/O infrastructure** — #67612 (Draft; synchronized with the current infrastructure components; validation is recorded below).
- `FileRangeCoalescer`: linear coalescing of ordered, disjoint intervals, constrained by gap, merged size, and read amplification. An individual input range may exceed the merge-size limit.
- `FileRangePlanner`: map requested intervals to coalesced reads, then optionally extend reads to complete sufficiently covered cache-block boundaries.
- `FileRangeReadScheduler`: exact asynchronous range reads on the existing `segment_prefetch_thread_pool`; query/BE resident-buffer budgets, task completion, cancellation, and shutdown. No separate scheduling thread or extra concurrency cap.
- Shared `S3FileReader` statistics are atomic for concurrent range reads, with existing concurrency/retry coverage. This safety dependency is included in PR 1; the hole-fill pools and their configuration belong to PR 2.
- `CachedRemoteFileReader`: reuse the remote-only-on-cache-miss path for `NO_WRITE` reads; reuse in-flight cache-write buffers when they fully cover a requested interval, including intervals spanning multiple blocks.
- `ExecEnv`/`QueryContext`: scheduler ownership, shared per-query read context, runtime enable switch, and configurable byte budgets.
- Shared `ReadAheadStatistics`/bvar definitions, planner byte/range counters, and scheduler submission/outcome/IO counters. Query/BE budget rejection and executor rejection are separate from failures of accepted reads; cumulative BE IO totals remain visible after the final Profile snapshot.
- Review focus: interval invariants and complexity; rejection and partial submission; buffer lifetime/accounting; query cancellation and executor lifetime. This PR does not connect scanner or PageIO callers yet.
- [ ] **PR 2 — Consumed-range cache writeback and background hole filling** — planned; depends on PR 1.
- Complete-block submission and owned-buffer handoff to `AsyncCacheWriteManager`, including low-priority spare-capacity admission.
- `RangeCacheWriteback`: split consumed ranges by cache-block boundaries and route complete versus partial blocks.
- Hole planning: take the complement of covered intervals inside one cache block, then apply the range-coalescing algorithm with hole-fill-specific limits. Hole-fill requests remain within that block.
- `PartialBlockWritebackManager`: queued-fragment merging, bounded memory, queued-task eviction, and mutable `hole_fill_workers_per_be=32` block workers. A single merged hole range is read directly on its block worker; multiple ranges are submitted to the shared remote-read pool (`hole_fill_remote_read_threads_per_be=64`, mutable). This pool limit excludes inline single-range reads.
- `hole_fill_merge_delay_ms=10` gives each partial block a minimum queued aggregation interval; `0` disables it. Measure from successful admission and preserve the timestamp when merging later fragments. Dynamic updates wake workers and affect existing queued tasks; active tasks continue unchanged. Reuse the existing queue and condition variable, wait for the earliest deadline/capacity retry, and retain memory limits and eviction.
- Each block worker reuses one concurrent token. Join all accepted GETs before writeback or failure cleanup; stop workers and release their tokens before shutting down the remote-read pool. Worker pools start with nonzero minimum thread counts.
- Reuse inflight buffers and cached/downloading blocks before admission and once when a worker starts the task. The combined probe runs outside the queue mutex. Once hole reads start, execute the planned GETs without per-GET cache probes; retain final cache-writer checks.
- Partial-block requests validate themselves and tasks own buffer allocation and remote reads; the manager retains queue, worker and lifecycle accounting. Track complete/partial writeback, queued/active hole-fill capacity, remote requests/bytes/time, and aggregate failures. Review focus: task ownership, locking, pressure handling, and complete-block handoff. Cache population remains best effort.
- [ ] **PR 3 — Regular segment PageIO read-ahead** — planned; depends on PRs 1–2.
- `ColumnReadAhead` byte windows and `SegmentReadAhead` coordination, wired into column readers, PageIO, and the regular `SegmentIterator` path.
- After segment pruning, select candidate physical pages once and initialize all resolvable eager/lazy windows together. `read_ahead_eager_window_bytes=4 MiB` and `read_ahead_lazy_window_bytes=128 KiB` control compressed bytes per window. The second selected page triggers the next window; a one-page window uses its only page. Advance from the already resolved physical page index before Page Cache lookup, preserving ascending batches within reverse scans. First access and a large jump share the same initialization. Exact-row point and dependent nested-column reads group sorted row IDs by distinct pages.
- Consume completed range slices, retain synchronous fallback, and submit eligible consumed ranges for writeback only when the range actually read remote bytes. Pure File Cache/inflight hits avoid background hole filling.
- Record compressed candidate/Page Cache bytes, consumed-page/range bytes, ready/wait hits and data-page fallback attempts/bytes/time; flush deltas into the scanner Profile. Index/dictionary reads and Page Cache hits are excluded from fallback counters.
- `ColumnReadAheadPlan` owns reset and per-call timing aggregation. `ReadAheadColumnInitTime`, `ReadAheadPageAdvanceTime`, and `ReadAheadCurrentBatchPlanTime` distinguish initialization, scan advancement, and exact-row work; window-maintenance timers remain sub-timers.
- Review focus: initial candidate selection, page/window progression, eager/lazy dependencies, page lifetime, fallback, and avoidance of writes for unused speculative ranges.
- [ ] **PR 4 — Point-query data-page read-ahead** — planned; depends on PR 3.
- Include reusable exact-row page selection/preparation helpers and their tests with the point-query integration, so the component is self-contained against PR 3.
- Plan the row-store column or missing column-store values before decoding; retain row-cache handling, result padding, synchronous fallback, and request-scoped cleanup.
- Append point-query Profile counters to existing sampled diagnostics; use BE cumulative IO totals for unsampled workload deltas.
- TopN/global delayed-materialization second-stage integration and its RPC/Profile additions are outside the current PRs.

## Source commit mapping

Pick each row in the order shown. Follow-ups are folded into their owning components; intermediate commits introduce the final APIs. PR 1 is the exact five-commit prefix of #67292. Complete-block handoff/admission are consolidated, and scheduler-before-executor teardown is established in runtime ownership. TopN second-stage integration stays excluded.

| PR | Commits from #67292 |
| --- | --- |
| 1 | [e0205d32d15](https://github.com/apache/doris/pull/67292/commits/e0205d32d15e54285e6bcbaa5c6308c3f84ecd34) range coalescer → [aa6c8651faf](https://github.com/apache/doris/pull/67292/commits/aa6c8651fafbb45c6e7a7477e47e4a06fd647f89) cache-aware planner and shared metrics → [e86ee26fa90](https://github.com/apache/doris/pull/67292/commits/e86ee26fa90493ccc9dd2ce572d5b87c403732ea) asynchronous range scheduler → [6cea8dccf14](https://github.com/apache/doris/pull/67292/commits/6cea8dccf145e1916e126047c973fef42418ef14) exact no-write cached reads → [cdec0499937](https://github.com/apache/doris/pull/67292/commits/cdec0499937f782daf86d919fb1cb87674c67db3) query runtime ownership |
| 2 | [814e74bcecf](https://github.com/apache/doris/pull/67292/commits/814e74bcecf8acc2a98cff5840d1787ffdc51647) complete-block handoff and admission → [f1ebcd27b19](https://github.com/apache/doris/pull/67292/commits/f1ebcd27b19d6882623470cb258af9370e7992f8) cache-block range splitting → [e0b15d37409](https://github.com/apache/doris/pull/67292/commits/e0b15d37409977149f07b07e4aa951e61d35cd65) hole-fill planner → [ee76e426d2d](https://github.com/apache/doris/pull/67292/commits/ee76e426d2d9f59b4cae3fe7d9fa76885e1f6257) partial-block writeback manager → [7fc9227ba67](https://github.com/apache/doris/pull/67292/commits/7fc9227ba6710d48ed0181766ff5a38e13dc6b3b) hole-fill runtime wiring → [1cb4378f2b6](https://github.com/apache/doris/pull/67292/commits/1cb4378f2b6197b85034e45f1bc63039b2ba65a6) consumed-range writeback |
| 3 | [8d4edb506f7](https://github.com/apache/doris/pull/67292/commits/8d4edb506f74cfe0ba1443887a814e80b1f602a3) page-driven column windows → [0106540dedf](https://github.com/apache/doris/pull/67292/commits/0106540dedfbb713e2d7657973dc6ee87aa33136) segment coordination / pageio → [1b2f4146d0c](https://github.com/apache/doris/pull/67292/commits/1b2f4146d0c41ebb93746be52a148fe13d418c03) column-reader integration → [157327a3043](https://github.com/apache/doris/pull/67292/commits/157327a3043cfe6688a4cfc17c20e210456e01eb) regular scanner initialization → [d5093f87d39](https://github.com/apache/doris/pull/67292/commits/d5093f87d39f1468bbed85757750ff39109c0659) query writeback wiring |
| 4 | [270a7dadbb5](https://github.com/apache/doris/pull/67292/commits/270a7dadbb529b68c4f920cd3f3eca5b845b2cff) point queries and exact-row helpers |

## Validation and completion criteria

| Branch / PR | Current head | Validation for this update |
| --- | --- | --- |
| Validation #67579 | [d5c427fe337](https://github.com/apache/doris/pull/67579/commits/d5c427fe3372e6c65c8f2041dae6a3ee91d3f0d2) | ASAN/PCH: 198 tests / 11 suites passed |
| Full implementation #67292 | [270a7dadbb5](https://github.com/apache/doris/pull/67292/commits/270a7dadbb529b68c4f920cd3f3eca5b845b2cff) | Independent ASAN/PCH: 198 tests / 11 suites passed on the identical final source tree |
| Infrastructure #67612 | [cdec0499937](https://github.com/apache/doris/pull/67612/commits/cdec0499937f782daf86d919fb1cb87674c67db3) | Independent ASAN/PCH: 72 tests / 8 suites passed |

All runs use `run-be-ut.sh -j100`, ASAN and PCH, without a clean rebuild. Full-tree validation ran at `92dd0c2d85e`; moving the existing scheduler teardown-order fix into its runtime component changed commit history only, and the final tree is byte-identical. The tests cover page-driven forward/reverse scans, Page Cache progression, exact-row preparation, plan reuse and statistics, independent-window buffer reuse, asynchronous reads and hole filling. Formatting, BE build hygiene and diff checks pass. No cluster, external-object-store workload or clang-tidy was run; these tests do not establish a workload speedup.

Scope comparisons confirm that the full and validation trees differ only in the validation-only no-PCH compile fix. The final full tree is unchanged by history reorganization. The infrastructure update changes only six scheduler/statistics source/test files relative to its previous head; existing teardown ordering is preserved. TopN-specific paths remain identical to the shared base.

Earlier validation results (before TopN removal)

Previous parallel-hole-read synchronization from [1cde48989e8](https://github.com/apache/doris/pull/67579/commits/1cde48989e8b2ad81682529d0e6ece7ec99484b1):

| Target | Then-tested head | Synchronized scope | ASAN/PCH result |
| --- | --- | --- | --- |
| Full implementation #67292 | `dd0f30afabf` | Complete change folded into scheduler/S3 statistics, partial-block manager, and runtime configuration; 20 component commits | 84 existing tests / 5 suites passed |
| Infrastructure #67612 | `43bdd7d3c7c` | Shared S3 statistics safety and existing tests folded into scheduler; 5 component commits | 71 existing tests / 8 suites passed |
| Validation #67579 | `1cde48989e8` | Source change already present; commit correspondence updated; compile-only fix stays here | Source branch unchanged during this synchronization |

Both target branches were independently built and tested with `run-be-ut.sh -j100`, ASAN and PCH, without a clean rebuild. No new tests were added. Formatting, build hygiene and source-scope comparisons passed. The full implementation and validation trees differ only in the validation-only `err_utils.cpp` compile fix. No cluster, external-object-store workload or clang-tidy was run.

Earlier 2026-09-11 rebase and window synchronization (historical tested heads):

| Branch / PR | Tested head | ASAN/PCH result |
| --- | --- | --- |
| Full implementation #67292 | `e5b7da5e0df` | 255 tests / 17 suites passed |
| Infrastructure #67612 | `ea4107d6326` | 69 tests / 7 suites passed |
| Validation #67579 | `25eef4d68ff` | 46 tests / 4 suites passed |

All runs used `run-be-ut.sh -j100`, without a clean rebuild. Logs contain no ASAN/LSAN or runtime-error reports.

The typed access-path/read-ahead helper conflict in `column_reader_test.cpp` preserves both changes. Full-branch tests include the entire ColumnReader suite. Formatting, BE build hygiene and `git diff --check` pass. No cluster, external-object-store workload, or clang-tidy was run for this update.

Earlier validation (2026-09-10): the complete branch at `522d1a414a2` passed 184 focused tests from 17 suites; PR 1 at `f159cd25c09` independently passed 69 tests from 7 suites. Both used ASAN/PCH and `run-be-ut.sh -j100`, with formatting and BE build-hygiene checks passing. No cluster or external-object-store workload was run for this refresh.

- Each PR builds independently against its declared base and passes focused ASAN unit tests for its included components, formatting checks, and BE build-hygiene checks.
- PR 1 covers coalescing/planning boundaries, oversized inputs, budgets, executor rejection, cancellation, shutdown, exact cache reads, fully covered in-flight-buffer reads, accepted-task accounting, concurrent S3 reader statistics, and BE IO completion after the final Profile report.
- PR 2 covers block splitting, hole complements/coalescing, fragment merging, minimum aggregation time, unchanged admission timestamps on merge, live delay updates, pressure/eviction, block-worker and remote-pool resizing, inline single-range reads, parallel multi-range reads, joining after failures or shutdown, complete-block write admission, cached/inflight/downloading reuse, and blocks that become cached while queued.
- PRs 3–4 cover one-time scan candidate selection, page-driven forward/reverse windows, Page Cache progression, independent-window buffer reuse, and exact-row point/nested reads, including disabled/enabled operation, fallback, and cleanup. Run local storage-compute separated import/query smoke and suitable self-contained regression cases after those paths are integrated.
- Keep measured results on the corresponding PRs. Close this issue only when all four PRs are merged and the integrated paths have been validated.

Start with the delivery plan and the referenced PRs #67612, #67579, and #67292, then review the FileRangeCoalescer, FileRangePlanner, FileRangeReadScheduler, ColumnReadAhead, and SegmentReadAhead components. Component tests are included in the PRs; the work is done only when the dependent infrastructure, cache writeback, regular PageIO read-ahead, and point-query read-ahead PRs are merged and validated.

Written by the indexing model from the issue text.