aws / aws/aws-cli

cli_history writes credential material (STS temp credentials, Authorization/X-Amz-Security-Token headers) to disk in cleartext

Open
#10,560 0 comments 0 reactions 1 assignee Claimed by @RyanFitzSimmonsAK View on GitHub
bug investigating p2
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

### Describe the bug

When `cli_history = enabled` is set (either in `~/.aws/config` or via the CLI history feature's other documented enablement paths), the AWS CLI records every HTTP request/response event to a local SQLite database at `~/.aws/cli/history/history.db`. Nothing in the write path redacts this data before it's persisted:

- `awscli/botocore/endpoint.py`'s `_do_get_response` records the outgoing request's full headers verbatim: `history_recorder.record('HTTP_REQUEST', {..., 'headers': request.headers, ...})`. This includes the `Authorization` header (the caller's SigV4-signed credential) and, for temporary/session credentials, the full `X-Amz-Security-Token` header.
- The same method records the full parsed API response body verbatim: `history_recorder.record('PARSED_RESPONSE', parsed_response)`.

For any API call whose response body **is** credential material — most notably `sts:AssumeRole` and `sts:GetSessionToken`, whose entire purpose is to hand back a live `AccessKeyId`/`SecretAccessKey`/`SessionToken` — this means those temporary credentials get written to disk in cleartext, in a file that persists until the retention window rotates it out.

The only filtering that exists anywhere in the `history` feature is `DetailedFormatter._SIG_FILTER` in `awscli/customizations/history/show.py`, which masks the derived SigV4 signature suffix — but only when a user runs `aws history show` (display-time only, not a write-time safeguard), and it does nothing for the actual credential fields (`SecretAccessKey`, `SessionToken`, `Authorization` header, `X-Amz-Security-Token` header).

The database file does get `chmod 0600` on creation (`DatabaseConnection._set_file_permissions`), which is a real mitigation against other local users on a well-behaved POSIX filesystem — but that's a best-effort attempt wrapped in `except OSError: LOG.debug(...)` with no user-facing warning on failure, and it does nothing to protect the file from being swept into backups, dotfile-sync tools, or a forensic disk image.

I checked and did not find this already reported (searched issues for "history credentials/secret", "cli_history secret access key", "history plaintext") or acknowledged anywhere (`CHANGELOG.rst`, no `SECURITY.md` in this repo).

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

This is not a regression — as far as I can tell from the code, this has been the behavior since the `history` feature (and its `DatabaseHistoryHandler`) was introduced; there is no evidence any redaction was ever added and later removed.

### Expected Behavior

Enabling `cli_history` should not cause live AWS credentials (the caller's own, or newly-issued temporary credentials returned by STS) to be persisted to local disk in cleartext.

### Current Behavior

Every field of API responses, and every request header, is written verbatim to the SQLite history database.

### Reproduction Steps

This drives the real, unmodified production classes (`DatabaseConnection`, `DatabaseRecordWriter`, `RecordBuilder`, `DatabaseHistoryHandler`, and the actual `HistoryRecorder.record()` used by `awscli/botocore/endpoint.py`) with a simulated `sts:AssumeRole` round trip, then inspects the resulting SQLite file directly, the same way an attacker with local file read access (or a copy of a backup/synced `~/.aws` directory) would:

```python
import os, sqlite3, sys, tempfile

REPO = ""
sys.path.insert(0, os.path.join(REPO, "awscli")) # makes `import botocore` resolve to the vendored copy
sys.path.insert(0, REPO)

from botocore.history import get_global_history_recorder
from awscli.customizations.history.db import (
DatabaseConnection, DatabaseRecordWriter, RecordBuilder, DatabaseHistoryHandler,
)

HISTORY_RECORDER = get_global_history_recorder()
history_db_path = os.path.join(tempfile.mkdtemp(), "history.db")

connection = DatabaseConnection(history_db_path)
writer = DatabaseRecordWriter(connection)
db_handler = DatabaseHistoryHandler(writer, RecordBuilder())
HISTORY_RECORDER.add_handler(db_handler)
HISTORY_RECORDER.enable()

# What awscli/botocore/client.py + endpoint.py record for a real `aws sts assume-role` call:
HISTORY_RECORDER.record("API_CALL", {"service": "sts", "operation": "AssumeRole", "params": {}})
HISTORY_RECORDER.record("HTTP_REQUEST", {
"method": "POST",
"headers": {
"Authorization": "AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/.../sts/aws4_request, ...",
"X-Amz-Security-Token": "FQoGZXIvYXdzEBOG...LONGTERM-SESSION-TOKEN...",
},
"url": "https://sts.amazonaws.com/",
"body": b"Action=AssumeRole&RoleArn=arn:aws:iam::123456789012:role/Example&RoleSessionName=demo",
})
HISTORY_RECORDER.record("PARSED_RESPONSE", {
"Credentials": {
"AccessKeyId": "ASIAVERYSECRETACCESSKEY",
"SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYVERYSECRETKEY",
"SessionToken": "IQoJb3JpZ2luX2VjEA0a...GENUINE-TEMP-SESSION-TOKEN...",
"Expiration": "2026-01-01T01:00:00Z",
},
"AssumedRoleUser": {"Arn": "arn:aws:sts::123456789012:assumed-role/Example/demo"},
})

# Inspect the raw file, as an attacker with local read access would:
conn = sqlite3.connect(history_db_path)
for event_type, payload in conn.execute("SELECT event_type, payload FROM records"):
print(event_type, payload)
```

### Current output

```
API_CALL {"service": "sts", "operation": "AssumeRole", "params": {}}
HTTP_REQUEST {"method": "POST", "headers": {"Authorization": "AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/.../sts/aws4_request, ...", "X-Amz-Security-Token": "FQoGZXIvYXdzEBOG...LONGTERM-SESSION-TOKEN...", ...}, ...}
PARSED_RESPONSE {"Credentials": {"AccessKeyId": "ASIAVERYSECRETACCESSKEY", "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYVERYSECRETKEY", "SessionToken": "IQoJb3JpZ2luX2VjEA0a...GENUINE-TEMP-SESSION-TOKEN...", ...}, ...}
```

Every credential value (`Authorization`, `X-Amz-Security-Token`, `SecretAccessKey`, `SessionToken`) is present verbatim in the DB file.

### Possible Solution

I have a fix ready and will open a PR shortly: a redaction layer in `awscli/customizations/history/db.py`'s `RecordBuilder.build_record` (CLI-owned code, not the vendored botocore engine, so the fix survives a future botocore vendor sync) that runs before every record is persisted:

- `HTTP_REQUEST`: redacts the `Authorization` and `X-Amz-Security-Token` header values.
- `API_CALL` params / `PARSED_RESPONSE`: recursively redacts any dict value whose key exactly matches a curated set of known credential/secret field names (`SecretAccessKey`, `SessionToken`, `SecretString`, `Password`, `PrivateKey`, etc). This is an **exact-name** match, not a substring match, so common non-secret fields like `NextToken`/`ContinuationToken`/`ClientToken` are not caught. It's deliberately broader than botocore's own shape-level `"sensitive"` model metadata: STS's own service model marks `Credentials.SecretAccessKey` sensitive but **not** `Credentials.SessionToken`, even though a session token is just as directly usable as a live credential — so relying solely on per-service model annotations being complete would still leak it.
- `HTTP_RESPONSE`: the raw body at that point is still an unparsed wire-format blob (`PARSED_RESPONSE` is the structured equivalent of the same data) and can't be safely redacted field-by-field without risking either missing an embedded secret or corrupting the payload, so it's replaced wholesale with a redaction marker.

`AccessKeyId` is intentionally left unredacted — it's an identifier, not a secret on its own (matching STS's own model), and has real debugging value.

### Additional Information/Context

This is not a hypothetical: `sts:AssumeRole`/`sts:GetSessionToken` responses literally *are* credential material by design, and this is a widely-used, documented, opt-in CLI feature (`cli_history`). I'm filing this publicly per the reporter's own judgment after considering private disclosure; a fix PR follows immediately.

### CLI version used

Reproduced against current `develop` (verified against the actual `awscli/botocore/endpoint.py`/`awscli/customizations/history/db.py` source).

### Environment details (OS name and version, etc.)

macOS (platform-independent bug — the recording/write path has no OS-specific logic)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.