aws / aws/aws-cli

JSONFileCache.__setitem__ not multi-process safe.

Open
#9,632 5 comments 0 reactions 0 assignees View on GitHub
bug p3
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

### Describe the bug

If multiple aws commands run at the same time and sets the `.aws/cli/cache/*.json`, the cache file could get corrupted (not a valid json)

We noticed corrupted `.aws/cli/cache/*.json` file.

1. cache key is always the same - sha1 of args (which is just role arn and serial number) (makes sense) [[code](https://github.com/aws/aws-cli/blob/2.27.63/awscli/botocore/credentials.py#L812-L813)]
2. `JSONFileCache.__setitem__` (is not multi-process/thread safe) just does `os.fdopen` [[code](https://github.com/aws/aws-cli/blob/2.27.63/awscli/botocore/utils.py#L3935-L3937)]. If multiple processes open the file and write to this file at the same time, it could get corrupted (not valid json)

```
2025-07-31 11:21:08,705 - MainThread - botocore.credentials - DEBUG - hii in _create_cache_key args={"RoleArn": "arn:aws:iam:::role/role", "SerialNumber": "arn:aws:iam:::mfa/"}
2025-07-31 11:21:08,705 - MainThread - botocore.credentials - DEBUG - hii argument_hash=
2025-07-31 11:21:14,022 - MainThread - botocore.credentials - DEBUG - hii type(self._cache)=
2025-07-31 11:21:14,022 - MainThread - botocore.utils - DEBUG - hiii __setitem__ = /Users//.aws/cli/cache/.json

2025-07-31 11:20:17,696 - MainThread - botocore.credentials - DEBUG - hii in _create_cache_key args={"RoleArn": "arn:aws:iam:::role/role", "SerialNumber": "arn:aws:iam:::mfa/"}
2025-07-31 11:20:17,696 - MainThread - botocore.credentials - DEBUG - hii argument_hash=<>
2025-07-31 11:20:22,562 - MainThread - botocore.utils - DEBUG - hiii __setitem__ = /Users//.aws/cli/cache/.json
```

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

`/Users//.aws/cli/cache/.json` to always have valid json

### Current Behavior

`/Users//.aws/cli/cache/.json` under some race condition can store invalid json

### Reproduction Steps

I dont have clear reproduction steps. Run many aws cli without the cache file.

### Possible Solution

Make `JSONFileCache.__setitem__` function multi process safe.

### Additional Information/Context

_No response_

### CLI version used

aws-cli/2.26.1

### Environment details (OS name and version, etc.)

Mac

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.