apache / apache/airflow

Allow backend DB to authenticate using temporary tokens

Open
#30,368 7 comments 3 reactions 0 assignees View on GitHub
area:core good first issue kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Based on [this](https://github.com/apache/airflow/discussions/30283) discussion.
Currrently there is no way to use token identity to authenticate with amazon RDS without a fairly significant change to the helm charts and airflow code.

I will implement this functionality and add the helm options as:

```yaml
externalDatabase:
type: postgres
host: airflow-cluster..us-east-1.rds.amazonaws.com

## the port of the external database
##
port: 5432

## the database/scheme to use within the external database
##
database: airflow

## the username for the external database
##
user: airflow

awsRdsTokenIdentity:
enabled: true
region: us-east-1
connectionExpirySeconds: 600
```

And use sqlalchemy envents to provide the token.
```python
def amend_connection(cparams):
if conf.getboolean("database", "use_aws_token_identity"):
log.info(f'connecting user {cparams["user"]} to {cparams["host"]}:{cparams["host"]} using pod identity')
client = boto3.client(
"rds",
region_name=conf.get_mandatory_value("database", "aws_region"),
)
token = client.generate_db_auth_token(
DBHostname=cparams["host"],
Port=cparams["port"],
DBUsername=cparams["user"],
)
cparams["password"] = token
else:
log.info(f'connecting {cparams["user"]} using user/password')

@event.listens_for(engine, "do_connect")
def provide_token(dialect, conn_rec, cargs, cparams):
amend_connection(cparams)

```

### Use case/motivation

Temporary credentials are a security feature generally required secops and a general good practice these days, so it makes sense for me to support them.

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with the Helm chart's externalDatabase options and Airflow's database configuration, then inspect the SQLAlchemy connection event entry point described in the issue. Verify how AWS RDS token generation should supply temporary credentials for PostgreSQL connections and how the region and expiry settings are represented. Done means token-authenticated connections work with the documented Helm options.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, helm, postgresql, python, sqlalchemy
Domain
authentication, backend, cloud, databases, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.