apache / apache/airflow

Secrets masker: non-string values are not redacted when the key name is sensitive

Open
#71,275 1 comment 0 reactions 0 assignees View on GitHub
area:core area:secrets kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.3.0

### What happened and how to reproduce it?

**Issue Description**

When a Variable's key name matches the sensitive-keyword list, `GET /api/v2/variables`
correctly returns `***` for string values, but returns non-string values (int, float,
bool) in cleartext.

`SecretsMasker._redact_all` is the fail-closed path used when a key name is judged
sensitive, but it only replaces `str`, recurses into containers, and returns every
other type unchanged:

```python
# airflow/_shared/secrets_masker/secrets_masker.py:323-347
def _redact_all(self, item, depth, max_depth=MAX_RECURSION_DEPTH, *, replacement="***"):
if depth > max_depth or isinstance(item, str):
return replacement
if isinstance(item, dict): ...
if isinstance(item, (tuple, set)): ...
if isinstance(item, list): ...
return item # <-- non-str scalars pass through unredacted
```

It is reached from `_redact` line 358:

```python
if name and self.should_hide_value_for_key(name):
return self._redact_all(item, depth, max_depth, replacement=replacement)
```

The comment at lines 353-355 states that key-name-based redaction "must fail closed at
any nesting level". It fails closed on depth, but not on type.

**Steps to reproduce**

1. Create Variables whose names contain a sensitive keyword:

```bash
airflow variables set test-password-a "abcd"
airflow variables set test-password-b "1234"
airflow variables set test-password-c "12ab"
airflow variables set test-password-d "1234.5"
airflow variables set test-password-e "true"
```

2. Read them back via the API (or view Admin -> Variables in the UI):

```bash
curl -H "Authorization: Bearer $TOKEN" \
"http:///api/v2/variables?limit=100"
```

3. Observed:

| value | returned |
|----------|-----------|
| `abcd` | `***` |
| `12ab` | `***` |
| `1234` | `1234` |
| `1234.5` | `1234.5` |
| `true` | `true` |

Values that are valid JSON scalars (int, float, bool) are returned unmasked; values
that remain strings are masked correctly.

**Not affected: task logs**

Task logs mask correctly for numeric values, because that path registers the secret
value as a substring pattern rather than going through `_redact_all`:

```python
@task
def leak_test():
from airflow.sdk import Variable
print("numeric:", Variable.get("test-password-num")) # 12345678 -> ***
print("string:", Variable.get("test-password-long")) # abcdefgh -> ***
```

Both are masked. (Use values of 5+ characters — shorter ones hit the
"Skipping masking for a secret as it's too short (<5 chars)" guard.)

The same code exists at the same line numbers in
`airflow/sdk/_shared/secrets_masker/secrets_masker.py`.

### What you think should happen instead?

Once `should_hide_value_for_key(name)` returns True, the value should be redacted
regardless of its Python type. A numeric PIN, account number, or all-digit API key
stored under a `*_password` / `*_token` key is a realistic case, and the current
behaviour silently returns it in cleartext while an equivalent alphanumeric value
is masked.

A possible fix is to invert the type check in `_redact_all` so that containers are
traversed and everything else is replaced:

```python
if depth > max_depth or not isinstance(item, (dict, tuple, set, list)):
return replacement
```

I'm not sure whether non-strings are deliberately preserved for `merge()`, which
restores original values where `***` is unchanged — a maintainer should confirm
that before applying anything like the above.

### Operating System

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian

### Deployment

Docker-Compose

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

Based on apache/airflow:3.3.0, with apache-airflow-providers-keycloak==0.8.2 installed. No changes to core Airflow.

### Anything else?

ccurs every time. Deployment uses the Keycloak auth manager, but the code path is in core and is not auth-manager-specific. Python 3.13.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start in airflow/_shared/secrets_masker/secrets_masker.py at SecretsMasker._redact_all and _redact; compare the same implementation in airflow/sdk/_shared/secrets_masker/secrets_masker.py. Confirm how merge() handles replacements, then ensure key-name-based redaction masks non-string scalar values while still traversing containers and respecting the existing depth behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.