[Feature Request]: Support reading from BigTable Continuous Materialized Views in BigtableIO
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What would you like to happen?
The BigTable connector via `BigtableIO.Read` currently only supports reading from BigTable tables via `withTableId()`. There is no way to read from the new [continuous materialized view tables](https://cloud.google.com/bigtable/docs/continuous-materialized-views) (CMV), which are a separate resource type (`materializedViews/` vs `tables/`).
The underlying google-cloud-bigtable Java client already supports CMVs — `Query.create()` and sampleRowKeys() both accept materialized view resource names as of [v2.55.0](https://github.com/googleapis/java-bigtable/releases/tag/v2.55.0). However, `BigtableIO.Read `has no option to specify a materialized view name, so this capability is not accessible from Beam pipelines.
**Use case:**
We use BigTable CMVs to maintain pre-aggregated index views over our data. We need to read from these views in Dataflow pipelines for downstream processing. Currently we can only access CMVs via the REST API or the Java client directly, but not through BigtableIO.
**Expected behavior:**
A `withMaterializedViewName(String)` method (or similar) on `BigtableIO.Read`, mutually exclusive with withTableId(), that routes the read through the existing client support for materialized views.
### Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
### Issue Components
- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [x] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
Contributor guide
Assessment
This issue has not been assessed yet.