ClickHouse / ClickHouse/pg_clickhouse

Add support for asynchronous fetching, the `async_capable` table option

Open
#294 1 comment 0 reactions 0 assignees View on GitHub
enhancement planner
Dominant language
C
Stars
283
Forks
21
Avg merge
2d 6h
Merged PRs (30d)
11

Description

Add support for the `async_capable` option to allow async fetching of result sets, as described in [All Your GUCs in a Row: `enable_async_append`](https://thebuild.com/blog/all-your-gucs-in-a-row-enableasyncappend/) and the [FDW Routines for Asynchronous Execution](https://www.postgresql.org/docs/18/fdw-callbacks.html#FDW-CALLBACKS-ASYNC) docs. The option was added to postgres_fdw in postgres/postgres@27e1f1456 and modified in postgres/postgres@bb684c82 and postgres/postgres@5c854e7.

This will allow Postgres to execute fetches via pg_clickhouse asynchronously with other foreign tables, via pg_clickhouse or otherwise. Note, however, that its utility will be limited for fetches from the same foreign server, as pg_clickhouse uses a a single connection for all queries for a given server, so they'll run synchronously. Thus this feature, while desirable, is less important than improving join pushdown in general.

Suggested by @heavycrystal.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with PostgreSQL's postgres_fdw changes referenced in commits 27e1f1456, bb684c82, and 5c854e7, then read the FDW Routines for Asynchronous Execution documentation. Trace pg_clickhouse's foreign-table option handling and fetch entry points; done means async_capable is supported and fetches can execute asynchronously alongside other foreign tables.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, clickhouse, postgresql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.