Allow backend DB to authenticate using temporary tokens
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Description
Based on [this](https://github.com/apache/airflow/discussions/30283) discussion.
Currrently there is no way to use token identity to authenticate with amazon RDS without a fairly significant change to the helm charts and airflow code.
I will implement this functionality and add the helm options as:
```yaml
externalDatabase:
type: postgres
host: airflow-cluster..us-east-1.rds.amazonaws.com
## the port of the external database
##
port: 5432
## the database/scheme to use within the external database
##
database: airflow
## the username for the external database
##
user: airflow
awsRdsTokenIdentity:
enabled: true
region: us-east-1
connectionExpirySeconds: 600
```
And use sqlalchemy envents to provide the token.
```python
def amend_connection(cparams):
if conf.getboolean("database", "use_aws_token_identity"):
log.info(f'connecting user {cparams["user"]} to {cparams["host"]}:{cparams["host"]} using pod identity')
client = boto3.client(
"rds",
region_name=conf.get_mandatory_value("database", "aws_region"),
)
token = client.generate_db_auth_token(
DBHostname=cparams["host"],
Port=cparams["port"],
DBUsername=cparams["user"],
)
cparams["password"] = token
else:
log.info(f'connecting {cparams["user"]} using user/password')
@event.listens_for(engine, "do_connect")
def provide_token(dialect, conn_rec, cargs, cparams):
amend_connection(cparams)
```
### Use case/motivation
Temporary credentials are a security feature generally required secops and a general good practice these days, so it makes sense for me to support them.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the Helm chart's externalDatabase options and Airflow's database configuration, then inspect the SQLAlchemy connection event entry point described in the issue. Verify how AWS RDS token generation should supply temporary credentials for PostgreSQL connections and how the region and expiry settings are represented. Done means token-authenticated connections work with the documented Helm options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, helm, postgresql, python, sqlalchemy
- Domain
- authentication, backend, cloud, databases, devops
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100