apache / apache/beam

[Feature Request]: Add read() support to Java ClickHouseIO (currently write-only)

Open
#40,048 1 comment 0 reactions 1 assignee Claimed by @nitinware View on GitHub
io java new feature P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

## Motivation

The Java `ClickHouseIO` connector at `sdks/java/io/clickhouse/` currently supports **only writes**. Reading from ClickHouse in a Beam pipeline requires falling back to `JdbcIO` with the ClickHouse JDBC driver, which loses ClickHouse-specific schema fidelity (`LowCardinality`, `Array(T)`, `Nullable(T)`, `Nested`, tuples) and native columnar transport.

## Proposal

Add `ClickHouseIO.read()` and `ClickHouseIO.readAll()` PTransforms symmetric to the existing write path:

- Native `Row` output using the ClickHouse client's native format
- Query/projection via user-supplied `SELECT` template
- Splittable reader for parallel reads across partitions / shards,
modeled on `JdbcIO`'s splitting pattern
- Reuse the existing connection configuration builder from the writer

## Benefits

- Closes an obvious read/write asymmetry — every other major DB IO in
Beam (JDBC, Cassandra, Mongo, HBase, Spanner, BigQuery, Snowflake)
supports both directions
- ClickHouse is increasingly common as an analytics source; users
currently work around this with `JdbcIO` at the cost of type fidelity

## Testing plan

- Unit tests mirroring the existing writer test file
- Integration test using the module's existing testcontainers-based
ClickHouse setup (`ClickHouseContainer` is already wired in `build.gradle`)
- No new dependencies required

### Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

### Issue Components

- [ ] Component: Python SDK
- [x] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [x] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Prism Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.