apache / apache/airflow

Add additional authentication for Google Cloud Operators

Open
#35,899 3 comments 1 reaction 0 assignees View on GitHub
area:providers kind:feature provider:google
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Currently, the only way to authenticate with Google Cloud operators is through defining a file path to the Google Cloud Credentials. This is not always a safe option, and credentials can be acquired in different ways through the Workload Federated Identity service in Google Cloud Platform.

This can be done in the following:
- Request a token from your external service (defined in your Federated Identity)
- Provide the following items as the payload to this api endpoint: https://sts.googleapis.com/v1/token
- subject_token: the token from the previous request
- grant_type: "urn:ietf:params:oauth:grant-type:token-exchange"
- requested_token_type: "urn:ietf:params:oauth:token-type:access_token"
- scope: "https://www.googleapis.com/auth/cloud-platform"
- subject_token_type: "urn:ietf:params:oauth:token-type:jwt"
- Provide the following items to this api endpoint: https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/:generateAccessToken
- Headers: use the token from the previous request in the "Authorization" section
- Payload:
- scope: ["https://www.googleapis.com/auth/cloud-platform"]
- Use the token from the last request as the authentication.

### Use case/motivation

I would like to be able to authenticate the Google Cloud operators through the Workload Federated Identity instead of providing the file path to credentials. This would help me to be able to use Airflow's native Google Cloud operators instead of making custom operators.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Review the Google Cloud operators' current credentials-file authentication and the Workload Federated Identity flow described in the issue, including the Google STS and IAM Credentials endpoints. Done means operators can authenticate through federated identity without a credentials file, with tests covering the new authentication path.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
authentication, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.