apache / apache/beam

[Feature Request]: Provide option in ReadFromBigQuery to set project from which queries are executed/billed

Open
#26,904 1 comment 2 reactions 0 assignees View on GitHub
awaiting triage dataflow io new feature P2 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

The [ReadFromBigQuery](https://github.com/apache/beam/blob/v2.47.0/sdks/python/apache_beam/io/gcp/bigquery.py#L2483) method should provide an option to set the query execution project:

- similar to the `load_job_project_id ` parameter in [WriteToBigQuery](https://beam.apache.org/releases/pydoc/2.47.0/apache_beam.io.gcp.bigquery.html#apache_beam.io.gcp.bigquery.WriteToBigQuery) in the Python SDK.
-
- similar to the [`bigQueryProject`](https://beam.apache.org/documentation/io/built-in/google-bigquery/#:~:text=.of%28%29%29%29%3B-,Query%20execution%20project,-By%20default%20the) parameter in the Java SDK .

Currently, the project is [inferred](https://github.com/apache/beam/blob/v2.47.0/sdks/python/apache_beam/io/gcp/bigquery.py#L755-L765) from the project running the pipeline or the project which contains the temporary dataset, if the temp dataset is set.

### Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

### Issue Components

- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [X] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.