citusdata / citusdata/citus

PG19: evaluate async I/O and pg_stat_io adoption for columnar

Open
#8,773 2 comments 0 reactions 0 assignees View on GitHub
pg19_features
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

Part of #8597 (Phase 3 — later feature integration).

Covers two of the seven Phase 3 areas, because they are the same surface: columnar's buffer-read path. `pg_stat_io` only becomes meaningful for Citus if columnar adopts read streams.

## Async I/O — Citus currently gains nothing

PG19 makes async I/O the default: `DEFAULT_IO_METHOD` is `IOMETHOD_WORKER`, and `READ_STREAM_USE_BATCHING` is available. AIO benefits are delivered **through the read stream API** — there is no separate extension-facing AIO API to call.

Columnar does not use it on the path that matters:

- `src/backend/columnar/columnar_storage.c:673` — the genuine data-path read, via plain `ReadBuffer()`, inside `ReadFromBlock()`. **Correction (verified 2026-08-15):** this issue originally cited `:447,673,701` as read sites. That was wrong. `:447` is inside `ColumnarStorageReserveData()` and is `ReadBuffer(rel, P_NEW)` — a relation extend; `:701` is inside `WriteToBlock()`. Both are write path. Only `:673` is a read, so a read-stream conversion is one third the scope originally implied here.
- `src/backend/columnar/columnar_tableam.c:1470` — the only `read_stream_next_buffer()` call in the tree, and it is consumer-side only; columnar never calls `read_stream_begin_relation()`.

So columnar reads stay synchronous under PG19 defaults. This is **not a correctness problem** — nothing breaks, nothing is silently wrong. It is a throughput opportunity that heap access methods get for free and columnar does not.

Work would be: convert columnar's stripe/chunk read path to `read_stream_begin_relation()` + `read_stream_next_buffer()`, with `READ_STREAM_USE_BATCHING` where the access pattern is sequential. Needs measurement before commitment — columnar's access pattern may not benefit enough to justify the churn.

## pg_stat_io — no surface today

PG19 extends `pg_stat_io` with `read_bytes` / `write_bytes` / `extend_bytes`, adds `IOOBJECT_WAL`, and adds `pg_stat_get_backend_io()`. `pgstat_count_io_op*()` gained a `bytes` argument.

Citus has **zero references** to any of this in `src/` and in `src/test/` (verified with a recursive scan; a non-recursive glob gives a false negative here). Nothing to update and nothing to fix — columnar's I/O is invisible to `pg_stat_io` because it goes through the buffer manager as ordinary reads.

If and only if the read-stream conversion above happens, revisit whether columnar should report I/O with a distinguishable `IOObject` / `IOContext` so operators can attribute columnar I/O separately.

## Priority

Lowest of the Phase 3 items. Pure optimisation, no correctness dependency, does not gate the `pg19-support` → `main` merge. Should be measured before it is scheduled.

Contributor guide

Open the contributing guide

Research direction

Start with the read path at src/backend/columnar/columnar_storage.c:673 and compare it with the read-stream entry point in src/backend/columnar/columnar_tableam.c:1470. Measure columnar's sequential-read behavior under PG19 before deciding whether conversion is worthwhile; done means a documented performance decision, with pg_stat_io attribution considered only if read streams are adopted.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.