apache / apache/airflow

quota_project_id not exposed at operator level for Google Cloud operators

Open
#65,591 4 comments 0 reactions 0 assignees View on GitHub
area:providers kind:feature provider:google
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

Just noticed this while working on #56324 — ``quota_project_id ``was added to ``GoogleBaseHook`` but none of the Google Cloud operators expose it as a parameter. The only way to use it right now is through connection extras, which is connection-wide and can't be overridden per task. Every operator already does this pattern for ``impersonation_chain``,`` quota_project_id `` just needs the same treatment.
I'm planning to implement this; starting with BigQuery operators and then GCS, Dataflow, and Dataproc.
Happy to submit a PR for this! just wanted to check if there's any reason this pattern shouldn't be extended to operators before I start.

### Use case/motivation

if you have two BigQuery tasks in the same DAG that need to bill to different projects, there's currently no way to express that at the operator level. Organizations using shared service accounts across multiple billing projects hit this immediately.

### Related issues

#56324

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.