Unable to read from bigquery and publish to pubsub using dataflow runner (python SDK)
Open
dataflow
gcp
improvement
io
P3
python
runners
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
With the Python SDK:
The dataflow runner does not allow use of reading from bigquery in streaming pipelines.
Pubsub is not allowed for batch pipelines.
Thus, there's no way to create a pipeline on the dataflow runner that reads from bigquery and publishes to pubsub.
Imported from Jira [BEAM-8046](https://issues.apache.org/jira/browse/BEAM-8046). Original Jira may contain additional context.
Reported by: jimpremise.
Contributor guide
Assessment
This issue has not been assessed yet.