Secrets masker: non-string values are not redacted when the key name is sensitive
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.3.0
### What happened and how to reproduce it?
**Issue Description**
When a Variable's key name matches the sensitive-keyword list, `GET /api/v2/variables`
correctly returns `***` for string values, but returns non-string values (int, float,
bool) in cleartext.
`SecretsMasker._redact_all` is the fail-closed path used when a key name is judged
sensitive, but it only replaces `str`, recurses into containers, and returns every
other type unchanged:
```python
# airflow/_shared/secrets_masker/secrets_masker.py:323-347
def _redact_all(self, item, depth, max_depth=MAX_RECURSION_DEPTH, *, replacement="***"):
if depth > max_depth or isinstance(item, str):
return replacement
if isinstance(item, dict): ...
if isinstance(item, (tuple, set)): ...
if isinstance(item, list): ...
return item # <-- non-str scalars pass through unredacted
```
It is reached from `_redact` line 358:
```python
if name and self.should_hide_value_for_key(name):
return self._redact_all(item, depth, max_depth, replacement=replacement)
```
The comment at lines 353-355 states that key-name-based redaction "must fail closed at
any nesting level". It fails closed on depth, but not on type.
**Steps to reproduce**
1. Create Variables whose names contain a sensitive keyword:
```bash
airflow variables set test-password-a "abcd"
airflow variables set test-password-b "1234"
airflow variables set test-password-c "12ab"
airflow variables set test-password-d "1234.5"
airflow variables set test-password-e "true"
```
2. Read them back via the API (or view Admin -> Variables in the UI):
```bash
curl -H "Authorization: Bearer $TOKEN" \
"http:///api/v2/variables?limit=100"
```
3. Observed:
| value | returned |
|----------|-----------|
| `abcd` | `***` |
| `12ab` | `***` |
| `1234` | `1234` |
| `1234.5` | `1234.5` |
| `true` | `true` |
Values that are valid JSON scalars (int, float, bool) are returned unmasked; values
that remain strings are masked correctly.
**Not affected: task logs**
Task logs mask correctly for numeric values, because that path registers the secret
value as a substring pattern rather than going through `_redact_all`:
```python
@task
def leak_test():
from airflow.sdk import Variable
print("numeric:", Variable.get("test-password-num")) # 12345678 -> ***
print("string:", Variable.get("test-password-long")) # abcdefgh -> ***
```
Both are masked. (Use values of 5+ characters — shorter ones hit the
"Skipping masking for a secret as it's too short (<5 chars)" guard.)
The same code exists at the same line numbers in
`airflow/sdk/_shared/secrets_masker/secrets_masker.py`.
### What you think should happen instead?
Once `should_hide_value_for_key(name)` returns True, the value should be redacted
regardless of its Python type. A numeric PIN, account number, or all-digit API key
stored under a `*_password` / `*_token` key is a realistic case, and the current
behaviour silently returns it in cleartext while an equivalent alphanumeric value
is masked.
A possible fix is to invert the type check in `_redact_all` so that containers are
traversed and everything else is replaced:
```python
if depth > max_depth or not isinstance(item, (dict, tuple, set, list)):
return replacement
```
I'm not sure whether non-strings are deliberately preserved for `merge()`, which
restores original values where `***` is unchanged — a maintainer should confirm
that before applying anything like the above.
### Operating System
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian
### Deployment
Docker-Compose
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
Based on apache/airflow:3.3.0, with apache-airflow-providers-keycloak==0.8.2 installed. No changes to core Airflow.
### Anything else?
ccurs every time. Deployment uses the Keycloak auth manager, but the code path is in core and is not auth-manager-specific. Python 3.13.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start in airflow/_shared/secrets_masker/secrets_masker.py at SecretsMasker._redact_all and _redact; compare the same implementation in airflow/sdk/_shared/secrets_masker/secrets_masker.py. Confirm how merge() handles replacements, then ensure key-name-based redaction masks non-string scalar values while still traversing containers and respecting the existing depth behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100