apache / apache/airflow

Secrets from Airflow configuration are not masked in task logs

Open
#63,921 4 comments 0 reactions 0 assignees View on GitHub
area:core area:logging kind:bug needs-triage priority:high
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Apache Airflow version

3.1.8

### If "Other Airflow 3 version" selected, which one?

_No response_

### What happened?

Secrets from Airflow configuration are not masked in task logs if printed with stdlib or structlog.

This has to be culprit:
https://github.com/apache/airflow/blob/e3c10eae08cd8c0d28f1edaab24e11bd25606d6c/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L2094

### What you think should happen instead?

Secrets should be masked.

### How to reproduce

1. Run locally breeze
2. Deploy DAG

```
import datetime

from airflow import DAG
from airflow.configuration import conf
from airflow.providers.standard.operators.python import PythonOperator
import structlog

structlog_logger = structlog.get_logger()

def _print():
for section, key in [
("webserver", "secret_key"),
("api", "secret_key"),
("api_auth", "jwt_secret"),
]:
print("print", section, key, conf.get(section, key, fallback=None))
structlog_logger.info("structlog " + section + " " + key + " " + conf.get(section, key, fallback=""))

with DAG(
dag_id="print_secrets",
schedule=None,
start_date=datetime.datetime(2010, 1, 1),
catchup=False,
) as dag:
operator = PythonOperator(
dag=dag,
task_id="task",
python_callable=_print,
)
```
3. Observe secrets visible in the task logs (Airflow UI)
Example:

Image

### Operating System

Linux

### Versions of Apache Airflow Providers

_No response_

### Deployment

Docker-Compose

### Deployment details

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start at task-sdk/src/airflow/sdk/execution_time/supervisor.py around line 2094, then reproduce the issue with the provided DAG using local Breeze. Check both stdlib print output and structlog output in the task logs; done means Airflow configuration secrets such as webserver.secret_key and api.secret_key are masked in both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability, security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.