Secrets from Airflow configuration are not masked in task logs
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
3.1.8
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
Secrets from Airflow configuration are not masked in task logs if printed with stdlib or structlog.
This has to be culprit:
https://github.com/apache/airflow/blob/e3c10eae08cd8c0d28f1edaab24e11bd25606d6c/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L2094
### What you think should happen instead?
Secrets should be masked.
### How to reproduce
1. Run locally breeze
2. Deploy DAG
```
import datetime
from airflow import DAG
from airflow.configuration import conf
from airflow.providers.standard.operators.python import PythonOperator
import structlog
structlog_logger = structlog.get_logger()
def _print():
for section, key in [
("webserver", "secret_key"),
("api", "secret_key"),
("api_auth", "jwt_secret"),
]:
print("print", section, key, conf.get(section, key, fallback=None))
structlog_logger.info("structlog " + section + " " + key + " " + conf.get(section, key, fallback=""))
with DAG(
dag_id="print_secrets",
schedule=None,
start_date=datetime.datetime(2010, 1, 1),
catchup=False,
) as dag:
operator = PythonOperator(
dag=dag,
task_id="task",
python_callable=_print,
)
```
3. Observe secrets visible in the task logs (Airflow UI)
Example:
### Operating System
Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start at task-sdk/src/airflow/sdk/execution_time/supervisor.py around line 2094, then reproduce the issue with the provided DAG using local Breeze. Check both stdlib print output and structlog output in the task logs; done means Airflow configuration secrets such as webserver.secret_key and api.secret_key are masked in both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability, security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100